Skip to content

TIKA-4850: Emit audio cover art as a THUMBNAIL embedded document - #3090

Open
dschmidt wants to merge 12 commits into
apache:mainfrom
dschmidt:audio-cover-thumbnail
Open

TIKA-4850: Emit audio cover art as a THUMBNAIL embedded document#3090
dschmidt wants to merge 12 commits into
apache:mainfrom
dschmidt:audio-cover-thumbnail

Conversation

@dschmidt

Copy link
Copy Markdown
Contributor

The audio parsers emitted cover art as INLINE, while every document container parser marks its preview image THUMBNAIL. A client that wants the representative image of a file can now take the first THUMBNAIL for audio too. The front cover (ID3 APIC / FLAC / Vorbis picture type 3, or the first picture if there is none) is the thumbnail; further pictures stay INLINE. MP4 covr carries no picture type, so its first image is the thumbnail. The choice is shared in CoverArt; the FLAC/Vorbis picture blocks are parsed into PictureBlock first so the choice sees all of them. Clients that looked for cover art as INLINE need to accept THUMBNAIL as well (noted in CHANGES).

https://issues.apache.org/jira/browse/TIKA-4850

The front cover (ID3 APIC and FLAC/Vorbis picture type 3), or the first
picture if there is none, is the picture that stands for the file, so mark
it THUMBNAIL like the preview image of the document container formats; the
other pictures stay INLINE. MP4 covr carries no picture type, so its first
image is the thumbnail. The shared choice lives in CoverArt; the FLAC and
Vorbis picture blocks are parsed into PictureBlock first so the choice can
see all of them.
TikaMp4BoxHandler builds a TikaUserDataBox per udta box, so a per-box
count made the first cover of every box a thumbnail.
@dschmidt
dschmidt marked this pull request as ready for review August 29, 2026 08:55
@THausherr
THausherr requested a lite review from Copilot August 29, 2026 09:43

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates Tika’s audio parsers to emit the representative cover image as a THUMBNAIL embedded document (instead of INLINE), aligning audio behavior with container/document preview-image handling and enabling clients to consistently retrieve a “primary image” via the first THUMBNAIL.

Changes:

  • Introduces shared thumbnail-selection logic in CoverArt (front cover if present, else first image).
  • Updates MP3/OGG(FLAC+Vorbis)/MP4 parsers to mark the selected cover as THUMBNAIL and keep remaining images INLINE.
  • Updates/extends tests and documents the behavioral change in CHANGES.txt.

Reviewed changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
tika-parsers/.../src/test/java/org/apache/tika/parser/ogg/VorbisParserTest.java Updates assertions so the primary cover is THUMBNAIL and secondary covers remain INLINE.
tika-parsers/.../src/test/java/org/apache/tika/parser/ogg/OggAudioParserTest.java Adds coverage for thumbnail selection rules (single image vs. front-cover preference).
tika-parsers/.../src/test/java/org/apache/tika/parser/ogg/FlacParserTest.java Updates FLAC cover-art tests to assert THUMBNAIL for the representative image.
tika-parsers/.../src/test/java/org/apache/tika/parser/mp4/MP4ParserTest.java Updates MP4 covr cover-art tests to treat the first image as THUMBNAIL.
tika-parsers/.../src/test/java/org/apache/tika/parser/mp3/Mp3ParserTest.java Updates MP3 cover-art tests to assert the front cover is emitted as THUMBNAIL.
tika-parsers/.../src/main/java/org/apache/tika/parser/ogg/OggAudioParser.java Refactors Vorbis/FLAC picture-block handling to select and emit a single THUMBNAIL.
tika-parsers/.../src/main/java/org/apache/tika/parser/ogg/FlacParser.java Adjusts native-FLAC picture extraction to select a THUMBNAIL after collecting picture blocks.
tika-parsers/.../src/main/java/org/apache/tika/parser/mp4/boxes/TikaUserDataBox.java Marks first MP4 covr image as THUMBNAIL, remaining as INLINE.
tika-parsers/.../src/main/java/org/apache/tika/parser/mp3/Mp3Parser.java Emits ID3 pictures with a selected THUMBNAIL (front cover preferred).
tika-parsers/.../src/main/java/org/apache/tika/parser/audio/CoverArt.java Adds shared logic for choosing thumbnail index and mapping to embedded resource type.
CHANGES.txt Documents the behavior change for clients that previously relied on INLINE only.
Suppressed comments (1)

tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-audiovideo-module/src/main/java/org/apache/tika/parser/ogg/FlacParser.java:30

  • This import block contains a duplicate import java.util.List; and the java.* imports are out of order (e.g., java.nio.file.* interleaved with java.util.*). This commonly fails checkstyle rules around import grouping/order.
import java.nio.file.Path;
import java.util.Arrays;
import java.util.HashSet;
import java.util.List;
import java.util.Set;

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 12 out of 12 changed files in this pull request and generated no new comments.

Suppressed comments (4)

Previously missed (3) — in code that hasn't changed since the last review.

tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-audiovideo-module/src/test/java/org/apache/tika/parser/mp4/MP4ParserTest.java:204

  • getResourceAsStream(...) can return null; the current try-with-resources will throw a NullPointerException with an unhelpful stack trace if the test resource is missing or mispackaged. Add an explicit null check with a clear assertion message before reading bytes.
        byte[] file;
        try (InputStream is = getResourceAsStream("/test-documents/testMP4_coverArt.m4a")) {
            file = is.readAllBytes();
        }

tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-audiovideo-module/src/test/java/org/apache/tika/parser/mp4/MP4ParserTest.java:207

  • indexOf(...) returns -1 when the needle is not found; subtracting 4 and then wrapping a ByteBuffer at a negative offset will throw with a confusing exception. Add an assertion that the udta marker was found and that the computed box size is sane before slicing.
        //append a copy of the file's udta box (with its covr) at the top level
        int udta = indexOf(file, "udta".getBytes(StandardCharsets.ISO_8859_1)) - 4;
        int size = ByteBuffer.wrap(file, udta, 4).getInt();

tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-audiovideo-module/src/test/java/org/apache/tika/parser/mp4/MP4ParserTest.java:230

  • indexOf(...) currently allocates a new byte[] on every iteration via Arrays.copyOfRange, which is avoidable and can make this test unnecessarily slow/GC-heavy on larger inputs. A simple nested-loop compare avoids per-iteration allocations.
    private static int indexOf(byte[] haystack, byte[] needle) {
        for (int i = 0; i <= haystack.length - needle.length; i++) {
            if (Arrays.equals(Arrays.copyOfRange(haystack, i, i + needle.length), needle)) {
                return i;
            }

tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-audiovideo-module/src/main/java/org/apache/tika/parser/ogg/OggAudioParser.java:231

  • This implementation builds a List where each PictureBlock holds a byte[] copy of the image payload. That means all embedded image bytes are retained until after thumbnail selection, increasing peak heap usage to roughly the sum of all picture sizes (also affecting FLAC, which now collects PictureBlocks before calling extractPictures). For files with many/large embedded images, this can materially increase memory pressure and OOM risk.
        List<PictureBlock> pictures = new ArrayList<>();
        for (String block : comments.getComments(METADATA_BLOCK_PICTURE)) {
            byte[] decoded;
            try {
                decoded = Base64.getMimeDecoder().decode(block);

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants