TIKA-4850: Emit audio cover art as a THUMBNAIL embedded document - #3090
TIKA-4850: Emit audio cover art as a THUMBNAIL embedded document#3090dschmidt wants to merge 12 commits into
Conversation
The front cover (ID3 APIC and FLAC/Vorbis picture type 3), or the first picture if there is none, is the picture that stands for the file, so mark it THUMBNAIL like the preview image of the document container formats; the other pictures stay INLINE. MP4 covr carries no picture type, so its first image is the thumbnail. The shared choice lives in CoverArt; the FLAC and Vorbis picture blocks are parsed into PictureBlock first so the choice can see all of them.
TikaMp4BoxHandler builds a TikaUserDataBox per udta box, so a per-box count made the first cover of every box a thumbnail.
There was a problem hiding this comment.
Pull request overview
This PR updates Tika’s audio parsers to emit the representative cover image as a THUMBNAIL embedded document (instead of INLINE), aligning audio behavior with container/document preview-image handling and enabling clients to consistently retrieve a “primary image” via the first THUMBNAIL.
Changes:
- Introduces shared thumbnail-selection logic in
CoverArt(front cover if present, else first image). - Updates MP3/OGG(FLAC+Vorbis)/MP4 parsers to mark the selected cover as
THUMBNAILand keep remaining imagesINLINE. - Updates/extends tests and documents the behavioral change in
CHANGES.txt.
Reviewed changes
Copilot reviewed 11 out of 11 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| tika-parsers/.../src/test/java/org/apache/tika/parser/ogg/VorbisParserTest.java | Updates assertions so the primary cover is THUMBNAIL and secondary covers remain INLINE. |
| tika-parsers/.../src/test/java/org/apache/tika/parser/ogg/OggAudioParserTest.java | Adds coverage for thumbnail selection rules (single image vs. front-cover preference). |
| tika-parsers/.../src/test/java/org/apache/tika/parser/ogg/FlacParserTest.java | Updates FLAC cover-art tests to assert THUMBNAIL for the representative image. |
| tika-parsers/.../src/test/java/org/apache/tika/parser/mp4/MP4ParserTest.java | Updates MP4 covr cover-art tests to treat the first image as THUMBNAIL. |
| tika-parsers/.../src/test/java/org/apache/tika/parser/mp3/Mp3ParserTest.java | Updates MP3 cover-art tests to assert the front cover is emitted as THUMBNAIL. |
| tika-parsers/.../src/main/java/org/apache/tika/parser/ogg/OggAudioParser.java | Refactors Vorbis/FLAC picture-block handling to select and emit a single THUMBNAIL. |
| tika-parsers/.../src/main/java/org/apache/tika/parser/ogg/FlacParser.java | Adjusts native-FLAC picture extraction to select a THUMBNAIL after collecting picture blocks. |
| tika-parsers/.../src/main/java/org/apache/tika/parser/mp4/boxes/TikaUserDataBox.java | Marks first MP4 covr image as THUMBNAIL, remaining as INLINE. |
| tika-parsers/.../src/main/java/org/apache/tika/parser/mp3/Mp3Parser.java | Emits ID3 pictures with a selected THUMBNAIL (front cover preferred). |
| tika-parsers/.../src/main/java/org/apache/tika/parser/audio/CoverArt.java | Adds shared logic for choosing thumbnail index and mapping to embedded resource type. |
| CHANGES.txt | Documents the behavior change for clients that previously relied on INLINE only. |
Suppressed comments (1)
tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-audiovideo-module/src/main/java/org/apache/tika/parser/ogg/FlacParser.java:30
- This import block contains a duplicate
import java.util.List;and the java.* imports are out of order (e.g., java.nio.file.* interleaved with java.util.*). This commonly fails checkstyle rules around import grouping/order.
import java.nio.file.Path;
import java.util.Arrays;
import java.util.HashSet;
import java.util.List;
import java.util.Set;
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 12 out of 12 changed files in this pull request and generated no new comments.
Suppressed comments (4)
Previously missed (3) — in code that hasn't changed since the last review.
tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-audiovideo-module/src/test/java/org/apache/tika/parser/mp4/MP4ParserTest.java:204
- getResourceAsStream(...) can return null; the current try-with-resources will throw a NullPointerException with an unhelpful stack trace if the test resource is missing or mispackaged. Add an explicit null check with a clear assertion message before reading bytes.
byte[] file;
try (InputStream is = getResourceAsStream("/test-documents/testMP4_coverArt.m4a")) {
file = is.readAllBytes();
}
tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-audiovideo-module/src/test/java/org/apache/tika/parser/mp4/MP4ParserTest.java:207
- indexOf(...) returns -1 when the needle is not found; subtracting 4 and then wrapping a ByteBuffer at a negative offset will throw with a confusing exception. Add an assertion that the udta marker was found and that the computed box size is sane before slicing.
//append a copy of the file's udta box (with its covr) at the top level
int udta = indexOf(file, "udta".getBytes(StandardCharsets.ISO_8859_1)) - 4;
int size = ByteBuffer.wrap(file, udta, 4).getInt();
tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-audiovideo-module/src/test/java/org/apache/tika/parser/mp4/MP4ParserTest.java:230
- indexOf(...) currently allocates a new byte[] on every iteration via Arrays.copyOfRange, which is avoidable and can make this test unnecessarily slow/GC-heavy on larger inputs. A simple nested-loop compare avoids per-iteration allocations.
private static int indexOf(byte[] haystack, byte[] needle) {
for (int i = 0; i <= haystack.length - needle.length; i++) {
if (Arrays.equals(Arrays.copyOfRange(haystack, i, i + needle.length), needle)) {
return i;
}
tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-audiovideo-module/src/main/java/org/apache/tika/parser/ogg/OggAudioParser.java:231
- This implementation builds a List where each PictureBlock holds a byte[] copy of the image payload. That means all embedded image bytes are retained until after thumbnail selection, increasing peak heap usage to roughly the sum of all picture sizes (also affecting FLAC, which now collects PictureBlocks before calling extractPictures). For files with many/large embedded images, this can materially increase memory pressure and OOM risk.
List<PictureBlock> pictures = new ArrayList<>();
for (String block : comments.getComments(METADATA_BLOCK_PICTURE)) {
byte[] decoded;
try {
decoded = Base64.getMimeDecoder().decode(block);
# Conflicts: # CHANGES.txt
# Conflicts: # CHANGES.txt
# Conflicts: # CHANGES.txt
The audio parsers emitted cover art as INLINE, while every document container parser marks its preview image THUMBNAIL. A client that wants the representative image of a file can now take the first THUMBNAIL for audio too. The front cover (ID3 APIC / FLAC / Vorbis picture type 3, or the first picture if there is none) is the thumbnail; further pictures stay INLINE. MP4
covrcarries no picture type, so its first image is the thumbnail. The choice is shared inCoverArt; the FLAC/Vorbis picture blocks are parsed intoPictureBlockfirst so the choice sees all of them. Clients that looked for cover art as INLINE need to accept THUMBNAIL as well (noted in CHANGES).https://issues.apache.org/jira/browse/TIKA-4850