Skip to content

Null-terminate PooledBuffer and Utf8StringArray UTF-8 strings - #510

Open
René Larch (r-Larch) wants to merge 2 commits into
microsoft:mainfrom
r-Larch:fix/pooledbuffer-nul-terminator
Open

René Larch (r-Larch) wants to merge 2 commits into
microsoft:mainfrom
r-Larch:fix/pooledbuffer-nul-terminator

Conversation

@r-Larch

Copy link
Copy Markdown

Fixes #508. Also fixes #481 (same change as #482, in a separate commit that can be dropped if #482 lands first).

  • PooledBuffer.FromStringUtf8 / FromSpanUtf8: write the NUL terminator explicitly. On .NET the buffer comes from ArrayPool<byte>.Shared, which is not zeroed, so napi_throw_*_error, node_embedding_runtime_config_add_module and node_embedding_last_error_message_set read stale bytes past the string.
  • An empty string is encoded as just the terminator instead of PooledBuffer.Empty (null pointer), so JSError.ThrowError("") now actually throws. null still maps to a null pointer (used for the optional code argument).
  • EmbeddingRuntimeConfigAddModule: dispose the module-name buffer. Node.js copies the name into a std::string (EmbeddedRuntime::AddModule), so this is safe.
  • Utf8StringArray: write each terminator explicitly (Embedding args occasionally get trailing garbage on .NET (missing null terminator on a pooled buffer) #481). This is needed here because the new tests dirty the shared pool before the Node.js platform is created.

Tests

All fail on main and pass with the fix:

  • Utf8BufferTests: dirties the shared pool, then checks that both PooledBuffer helpers and Utf8StringArray produce NUL-terminated UTF-8 for "", ASCII and non-ASCII strings, and that null still pins to a null pointer.
  • NodejsEmbeddingTests.ThrowErrorFromCallback("x" / ""): throws from a .NET callback with a dirty pool and checks the JS e.message. Before: x followed by 15 garbage bytes, and "" did not throw at all.
  • New helper TestUtils.FillArrayPoolWithGarbage() (no-op on .NET Framework).

Local run (Windows x64, .NET SDK 10.0.402): net10.0 157/159, net9.0 126/127, net8.0 126/127. The only failure, napi-dotnet/overloads, fails the same way on main. I could not run net472 locally (the test-case builds fail with MSB4062 on this machine, also on main).

🤖 Generated with Claude Code

René Larch (r-Larch) and others added 2 commits October 21, 2026 21:01
The string buffer is rented from ArrayPool<byte>.Shared on .NET, which is not
zeroed, so the byte reserved for each terminator could hold stale data and
native code read argv entries past their end. Same fix as microsoft#482 (fixes microsoft#481);
needed here because the new tests dirty the shared pool before the Node.js
platform is created.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
PooledBuffer.FromStringUtf8 and FromSpanUtf8 reserved a byte for the
terminator but never wrote it. On .NET the array comes from
ArrayPool<byte>.Shared, which is not zeroed, so native functions that take a
null-terminated const char* without a length read stale bytes past the string
(or past the array): napi_throw_error / napi_throw_type_error /
napi_throw_range_error / node_api_throw_syntax_error (code and msg),
node_embedding_runtime_config_add_module and
node_embedding_last_error_message_set. For example JSError.ThrowError("x")
produced a JS error message of "x" followed by garbage, and a linked module
could be registered under a corrupted name.

- Write the terminator explicitly in both helpers.
- Encode an empty string as just the terminator instead of returning
  PooledBuffer.Empty, which pinned to a null pointer (ThrowError("") passed
  msg = NULL, so napi_throw_error failed and nothing was thrown). A null
  string still maps to a null pointer.
- Return the module-name buffer to the pool in
  EmbeddingRuntimeConfigAddModule (Node.js copies the name).

Fixes microsoft#508.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Copilot AI balanced review requested due to automatic review settings October 21, 2026 20:14
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The focused fixes have regression coverage, and static review identified no blocking issues.

No open issues 🎉

What changed in this PR

Fixes pooled UTF-8 string termination for native Node.js calls, addressing #508 and #481.

Changes:

  • Explicitly writes NUL terminators and distinguishes empty strings from null.
  • Disposes the module-name buffer after registration.
  • Adds regression tests using dirty pooled buffers.
File Description
test/​Utf8BufferTests.cs Tests UTF-8 termination and null handling.
test/​TestUtils.cs Adds a helper to dirty pooled buffers.
test/​NodejsEmbeddingTests.cs Tests callback error messages, including empty strings.
src/​NodeApi/​Runtime/​Utf8StringArray.cs Explicitly terminates each encoded string.
src/​NodeApi/​Runtime/​PooledBuffer.cs Terminates UTF-8 buffers and preserves null semantics.
src/​NodeApi/​Runtime/​NodejsRuntime.Embedding.cs Disposes the module-name buffer.

Review effort: Balanced

@r-Larch

Copy link
Copy Markdown
Author

René Larch (René Larch (@r-Larch)) please read the following Contributor License Agreement(CLA). If you agree with the CLA, please reply with the following information.

@microsoft-github-policy-service agree [company="{your company}"]

Options:

  • (default - no company specified) I have sole ownership of intellectual property rights to my Submissions and I am not making Submissions in the course of work for my employer.
@microsoft-github-policy-service agree
  • (when company given) I am making Submissions in the course of work for my employer (or my employer has intellectual property rights in my Submissions by contract or applicable law). I have permission from my employer to make Submissions and enter into this Agreement on behalf of my employer. By signing below, the defined term “You” includes me and my employer.
@microsoft-github-policy-service agree company="Microsoft"

Contributor License Agreement

@microsoft-github-policy-service agree

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

2 participants