Skip to content

feat!: richer, platform-identical metadata from getVideoMetaData and getImageMetaData - #418

Open
usama-liaqat wants to merge 4 commits into
numandev1:mainfrom
usama-liaqat:main
Open

feat!: richer, platform-identical metadata from getVideoMetaData and getImageMetaData#418
usama-liaqat wants to merge 4 commits into
numandev1:mainfrom
usama-liaqat:main

Conversation

@usama-liaqat

Copy link
Copy Markdown
Contributor

Summary

getVideoMetaData and getImageMetaData return far less than the platform APIs already hand them, and what they do return is shaped differently on iOS and Android β€” so every consumer ends up writing Platform.OS branches for basic facts like "is this video portrait?".

This widens both responses and makes them identical in shape across platforms. Everything added is read from the handle the code already opens: no extra decode, no second file read, no new native dependency beyond swapping Android's deprecated EXIF reader for the maintained AndroidX one.

Concretely, the problems this solves:

  • You could not tell a portrait video from a landscape one. Phone recordings store a landscape track plus a rotation flag, so a portrait clip reported 1920x1080 and comparing width to height gave the wrong answer. The rotation flag was never surfaced.
  • No bitrate, codec, HDR signal, or "does this have audio". Callers computed size * 8 / duration (which includes audio and container overhead), could not detect that re-encoding HEVC to H.264 may produce a larger file, could not decide "strip audio" in advance, and compressed HDR footage as if it were SDR β€” which comes out washed out and grey.
  • getImageMetaData returned NaN dimensions on Android for any image without EXIF (PNGs, screenshots, WebP, most edited images), because dimensions were read from the EXIF tags rather than the decoder. This was the most damaging bug in the old behavior.
  • The same fact had different key names and formats per platform β€” pixel dimensions, and GPS especially: a signed double plus a reference letter on one side, DMS rationals plus a reference letter on the other.

Four latent defects were found and fixed on the way:

  1. iOS getVideoMetaData could never settle. Audio-only files, and paths not beginning with file://, fell through every branch without resolving or rejecting β€” the caller's await hung forever with no error. Every path now settles.
  2. Android getVideoMetaData threw an NPE on audio-only and unusual files, because width/height/duration were read with a non-null assertion when those keys are legitimately absent.
  3. Android getVideoMetaData leaked its MediaMetadataRetriever β€” opened and never released on any path. Now released in a finally.
  4. Booleans reached JS as numbers on iOS. Nitro's AnyValue.fromAny matches an NSNumber as Double before it ever reaches its Bool case, so hasVideo: true arrived as 1. HybridCompressor now unwraps CFBoolean before handing values to Nitro. This affects every AnyMap result, including upload.

Breaking changes

  • getImageMetaData renames ImageWidth / ImageHeight / Orientation to width / height / orientation, and removes the raw exif passthrough. The old spellings are deliberately not kept as aliases: Orientation and orientation would differ only in case (invisible in review, one typo from reading undefined), and ImageWidth is precisely the field that returned NaN on Android β€” keeping it would ship the bug next to its fix. Every tag worth reading is now a normalized top-level key; a full oldβ†’new migration table is in the README.
  • getVideoMetaData removes the undeclared, Android-only creationTime key, which returned a raw "20240115T123456.000Z" string β€” or the literal string "null" when the file had no date.

These need a major version.

Notes for reviewers

  • Two fields are deliberately omitted on one platform rather than approximated, per "never invent a placeholder": frameCount is absent for video on iOS (reading it exactly needs an AVAssetReader pass over the samples; deriving duration Γ— frameRate would disagree with Android's exact frame-count), and isHDR is absent for images on Android, where detection is unreliable.
  • hasAlpha is read exactly on iOS but inferred from the container on Android (PNG/WebP/GIF), since confirming it would mean decoding the bitmap. Documented as such.
  • Android image metadata moves from the long-deprecated android.media.ExifInterface to androidx.exifinterface, which also fixes HEIC/WebP/PNG/DNG metadata on many devices and supplies rotation/mirroring/GPS conversion ready-made. This adds a small dependency.
  • The image path never decodes the bitmap β€” iOS keeps reading from a CGImageSource, and Android now does a bounds-only header decode β€” so a 100 MP image does not allocate a full bitmap.

Changelog

[GENERAL] [BREAKING] - getImageMetaData renames ImageWidth/ImageHeight/Orientation to width/height/orientation and removes the raw exif passthrough; see the README migration table
[GENERAL] [BREAKING] - getVideoMetaData removes the undeclared, Android-only creationTime key
[GENERAL] [ADDED] - getVideoMetaData returns displayWidth/displayHeight/rotation, hasVideo/hasAudio, isHDR/colorTransfer, codec, bitrate, frameRate, trackCount, fileName, parsed latitude/longitude, and the audio track's bitrate/sample rate/channels
[GENERAL] [ADDED] - getImageMetaData returns displayWidth/displayHeight/rotation/isMirrored, mimeType, hasAlpha, fileName, isAnimated/frameCount, bitDepth, colorSpace, dpiX/dpiY, signed GPS coordinates, and the camera fields (make, model, lensModel, software, iso, exposureTime, focalLength, flash)
[GENERAL] [ADDED] - Export the VideoMetaData, ImageMetaData and ColorTransfer response types from the package root
[IOS] [FIXED] - getVideoMetaData no longer hangs forever on audio-only files or paths not starting with file://; every path now resolves or rejects
[IOS] [FIXED] - Booleans no longer reach JS as numbers β€” HybridCompressor unwraps CFBoolean before handing values to Nitro, which affects every AnyMap result including upload
[ANDROID] [FIXED] - getImageMetaData no longer returns NaN dimensions for images without EXIF, and no longer rejects when the EXIF block is missing
[ANDROID] [FIXED] - getVideoMetaData no longer throws a null-pointer exception on audio-only files
[ANDROID] [FIXED] - getVideoMetaData no longer leaks its MediaMetadataRetriever
[ANDROID] [CHANGED] - Image metadata now uses androidx.exifinterface instead of the deprecated framework reader, fixing HEIC/WebP/PNG/DNG metadata on many devices
[GENERAL] [CHANGED] - The bare example's Image and Video screens display the full metadata for both the source and the compressed output
[INTERNAL] [CHANGED] - harness and __tests__ are now covered by yarn typecheck, and excluded from the published declarations

Test Plan

  • Ran local JS PR gate: yarn test:pr
  • If this changes compression/upload/download native behavior, ran the Harness tests on a simulator/device: yarn test:harness:android or yarn test:harness:ios.

yarn test:pr β€” exit 0

Test Suites: 1 passed, 1 total
Tests:       13 passed, 13 total
βœ– 6 problems (0 errors, 6 warnings)   # all pre-existing, in files this PR does not touch

yarn test:harness:ios β€” 9/9 passed on a booted iPhone 17 Pro simulator

HARNESS Runner ios ready
Test Suites: 1 passed, 1 total
Tests:       9 passed, 9 total

The harness assertions were extended to cover the new contract: rotation ∈ {0, 90, 180, 270}, display dimensions equal the stored ones swapped only on a quarter turn, and isHDR agrees with colorTransfer. Actual response for the video fixture:

{
  "width": 320, "height": 240, "displayWidth": 320, "displayHeight": 240, "rotation": 0,
  "hasVideo": true, "hasAudio": true, "isHDR": false, "colorTransfer": "sdr",
  "codec": "h264", "bitrate": 67144, "frameRate": 30, "trackCount": 2,
  "audioBitrate": 96906, "audioSampleRate": 44100, "audioChannels": 1,
  "duration": 2, "size": 44420, "extension": "mp4", "fileName": "compressor-harness-video.mp4"
}

This is what caught the hasVideo: 1 boolean bug β€” the type mismatch was invisible to the unit tests, which mock the native module entirely.

Native compile checks β€” both clean, with no new warnings in any touched native file:

yarn build:android   β†’ BUILD SUCCESSFUL
yarn build:ios       β†’ ** BUILD SUCCEEDED **

Bare example on a physical iPhone β€” built, installed and launched:

success Successfully built the app
success Installed the app on the device.
success Successfully launched the app

Not verified

yarn test:harness:android was not run. The APK builds and installs cannot proceed on the available device: a MIUI phone rejects every install path with INSTALL_FAILED_USER_RESTRICTED (gradle installDebug, adb install, and adb push + adb shell pm install all fail identically, since MIUI enforces this inside PackageManager), and the harness's emulator path cannot run while any physical device is attached because it calls emu avd name on every attached device.

So the Android side of this change is compile-verified only. The parts that most deserve a second pair of eyes, or a run on a working emulator:

  • the GIF/WebP frame counter in ImageMain.kt (a hand-written container walk β€” the newest, least-proven code here);
  • the NaN-dimensions fix via the bounds-only BitmapFactory read;
  • parseIso6709 and the AndroidX EXIF GPS conversion for southern/western coordinates, where a sign error cannot be caught by a northern-hemisphere fixture.

Widen getVideoMetaData and getImageMetaData to one shape across iOS and Android:
rotation, display dimensions, codec, bitrate, HDR, audio details, parsed GPS.

Fix an iOS promise that never settled on audio-only files, an Android NPE on the same,
a leaked MediaMetadataRetriever, NaN image dimensions without EXIF, and booleans
reaching JS as numbers on iOS.

BREAKING CHANGE: getImageMetaData renames ImageWidth/ImageHeight/Orientation to
width/height/orientation and drops the raw exif passthrough. See README migration table.

BREAKING CHANGE: getVideoMetaData drops the undeclared, Android-only creationTime key.
Drop the react-native-video patch: 6.19.2 ships the same fix upstream, replacing the
spread of StyleSheet.absoluteFillObject with StyleSheet.absoluteFill in Video.tsx.
Wraps install, pod install, the example run targets, native compile checks, tests and
the harness runs behind make targets.
Ruby version, Podfile.lock and signing team as written by bundle install, pod install
and a device build. Podfile.lock also catches react-native-compressor up to 2.0.3.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant