Repository navigation
Conversation
A valid zstd stream followed by a trailing skippable frame used to fail one-shot Decompress with "unexpected EOF": the streaming fallback stopped after the first frame and treated the skippable frame as truncated input. The behaviour changed in "Keep decompressing as long as we have compressed data", but no test covers the trailing-skippable shape specifically, and issue DataDog#129 remains open as a result. The test uses a highly compressible payload, since a large compressed input takes the direct path and never hits the fallback. It covers both the one-shot Decompress path and the streaming reader. Refs DataDog#129 Signed-off-by: Swarna Sekhar Dhar <bapay.9@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Refs #129
Why a test and no fix
The bug is real but already fixed on
1.xby "Keep decompressing as long as we have compressed data" (#155): that change made the streaming fallback keep consuming input after the first frame, so a trailing skippable frame no longer reads as truncated input. #129 was never closed, and nothing on1.xcovers the trailing-skippable shape — the two tests #155 added cover concatenated streams and truncated input, not this case.What the test pins down
TestDecompressTrailingSkippableFramecovers the one-shotDecompresspath with the exact shape from #129: a highly compressible 1,000,001-byte payload (so the compressed input is tiny and the streaming fallback runs), followed by a 12-byte skippable frame. That test fails withunexpected EOFon the parent of #155 and passes on current1.x.TestReaderTrailingSkippableFramecovers the same through the streaming reader with random data.Verified before opening:
The
go vetunsafe.Pointerwarnings inzstd_stream.goand thegofmtflag onzstd_stream_test.goare pre-existing and untouched by this change.If maintainers would rather #129 simply be closed as fixed by #155, that's fine too — happy to close this if so.