GH-112 Harden variant decoding: example data - #113
Conversation
Test files are added to parquet-format project with commentary.
e1eff0f to
6333d3f
Compare
steveloughran
changed the title
All parquet readers must reject this.
...from discussion in the iceberg PR.
| Robust implementations of variant decoders SHOULD reject these. | ||
|
|
||
| | File | Malformed Structure | | ||
| |---------------------------------------------------------------|----------------------------------------------------------------------------| |
There was a problem hiding this comment.
Would be great to add the sample code that helped generate these individual cases in the README as well.
There was a problem hiding this comment.
until the relevant PR is merged, that code doesn't really exist, does it?
There was a problem hiding this comment.
Agreed but those are in the tests of the Java repo. The data/README here has a snippet of the code that was used to generate the files. I'd prefer that be here rather than link to another repo. I'm okay reusing the same code but having it here is better for users to reproduce the artifacts. If you are planning to do that after the code merges, that's great.
There was a problem hiding this comment.
I think it is ok not to include reproducer of malformed data -- a link to the relevant PR (or perhaps a gist) would be enough
4 participants
Add parquet files with malformed variant data.
This is generated from the tests of apache/parquet-java#3562 when configured to not delete output during test runs