Skip to content

fix(ext/node): avoid trailing empty zstd frames - #36689

Open
ArakawaHenri wants to merge 2 commits into
denoland:mainfrom
ArakawaHenri:patch/zstd-empty-frame
Open

fix(ext/node): avoid trailing empty zstd frames#36689
ArakawaHenri wants to merge 2 commits into
denoland:mainfrom
ArakawaHenri:patch/zstd-empty-frame

Conversation

@ArakawaHenri

Copy link
Copy Markdown

Fixes #36688

ZstdCompress previously implemented an end write as run() followed by finish(). When the output buffer filled, the retry could insert ZSTD_e_continue into an in-progress ZSTD_e_end operation and append an empty frame.

This change:

  • calls ZSTD_compressStream2() once per write, preserving the requested continue, flush, or end directive across output chunks;
  • records when ZSTD_e_end completes while exactly filling the output buffer, preventing the following empty-input retry from starting another frame;
  • shares the corrected path between synchronous and asynchronous writes;
  • adds regressions for ordinary multi-chunk output and exact-fill completion.

Node parity

Node v26.7.0 passes its stored flush directive directly to ZSTD_compressStream2(). Matching that behavior restores Node parity for ordinary multi-chunk output.

Node's processChunk loop retries whenever availOutAfter === 0, so Node v26.7.0 currently appends an empty frame in the exact-fill case as well. This PR intentionally does not reproduce that boundary artifact: it depends only on internal output-buffer allocation and breaks consumers that validate exactly one frame. The return value from ZSTD_compressStream2() allows Deno to distinguish completed output from pending output.

The parity fix and the exact-fill guard are split into separate commits. If strict parity with Node's current exact-fill behavior is preferred, the first commit can be taken on its own.

Tests

  • ./tools/format.js --check
  • cargo check -p deno_node --features deno_core/v8
  • ./x test-node zstd
  • ./tools/lint.js

AI disclosure: ChatGPT assisted with bug investigation and patch implementation. I reviewed the changes and test results.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

node:zlib zstd compression appends trailing empty frame at output chunk boundaries

1 participant