feat!: richer, platform-identical metadata from getVideoMetaData and getImageMetaData - #418
Open
usama-liaqat wants to merge 4 commits into
Open
feat!: richer, platform-identical metadata from getVideoMetaData and getImageMetaData#418usama-liaqat wants to merge 4 commits into
usama-liaqat wants to merge 4 commits into
Conversation
Widen getVideoMetaData and getImageMetaData to one shape across iOS and Android: rotation, display dimensions, codec, bitrate, HDR, audio details, parsed GPS. Fix an iOS promise that never settled on audio-only files, an Android NPE on the same, a leaked MediaMetadataRetriever, NaN image dimensions without EXIF, and booleans reaching JS as numbers on iOS. BREAKING CHANGE: getImageMetaData renames ImageWidth/ImageHeight/Orientation to width/height/orientation and drops the raw exif passthrough. See README migration table. BREAKING CHANGE: getVideoMetaData drops the undeclared, Android-only creationTime key.
Drop the react-native-video patch: 6.19.2 ships the same fix upstream, replacing the spread of StyleSheet.absoluteFillObject with StyleSheet.absoluteFill in Video.tsx.
Wraps install, pod install, the example run targets, native compile checks, tests and the harness runs behind make targets.
Ruby version, Podfile.lock and signing team as written by bundle install, pod install and a device build. Podfile.lock also catches react-native-compressor up to 2.0.3.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
getVideoMetaDataandgetImageMetaDatareturn far less than the platform APIs already hand them, and what they do return is shaped differently on iOS and Android β so every consumer ends up writingPlatform.OSbranches for basic facts like "is this video portrait?".This widens both responses and makes them identical in shape across platforms. Everything added is read from the handle the code already opens: no extra decode, no second file read, no new native dependency beyond swapping Android's deprecated EXIF reader for the maintained AndroidX one.
Concretely, the problems this solves:
1920x1080and comparing width to height gave the wrong answer. The rotation flag was never surfaced.size * 8 / duration(which includes audio and container overhead), could not detect that re-encoding HEVC to H.264 may produce a larger file, could not decide "strip audio" in advance, and compressed HDR footage as if it were SDR β which comes out washed out and grey.getImageMetaDatareturnedNaNdimensions on Android for any image without EXIF (PNGs, screenshots, WebP, most edited images), because dimensions were read from the EXIF tags rather than the decoder. This was the most damaging bug in the old behavior.Four latent defects were found and fixed on the way:
getVideoMetaDatacould never settle. Audio-only files, and paths not beginning withfile://, fell through every branch without resolving or rejecting β the caller'sawaithung forever with no error. Every path now settles.getVideoMetaDatathrew an NPE on audio-only and unusual files, because width/height/duration were read with a non-null assertion when those keys are legitimately absent.getVideoMetaDataleaked itsMediaMetadataRetrieverβ opened and never released on any path. Now released in afinally.AnyValue.fromAnymatches anNSNumberasDoublebefore it ever reaches itsBoolcase, sohasVideo: truearrived as1.HybridCompressornow unwrapsCFBooleanbefore handing values to Nitro. This affects everyAnyMapresult, includingupload.Breaking changes
getImageMetaDatarenamesImageWidth/ImageHeight/Orientationtowidth/height/orientation, and removes the rawexifpassthrough. The old spellings are deliberately not kept as aliases:Orientationandorientationwould differ only in case (invisible in review, one typo from readingundefined), andImageWidthis precisely the field that returnedNaNon Android β keeping it would ship the bug next to its fix. Every tag worth reading is now a normalized top-level key; a full oldβnew migration table is in the README.getVideoMetaDataremoves the undeclared, Android-onlycreationTimekey, which returned a raw"20240115T123456.000Z"string β or the literal string"null"when the file had no date.These need a major version.
Notes for reviewers
frameCountis absent for video on iOS (reading it exactly needs anAVAssetReaderpass over the samples; derivingduration Γ frameRatewould disagree with Android's exactframe-count), andisHDRis absent for images on Android, where detection is unreliable.hasAlphais read exactly on iOS but inferred from the container on Android (PNG/WebP/GIF), since confirming it would mean decoding the bitmap. Documented as such.android.media.ExifInterfacetoandroidx.exifinterface, which also fixes HEIC/WebP/PNG/DNG metadata on many devices and supplies rotation/mirroring/GPS conversion ready-made. This adds a small dependency.CGImageSource, and Android now does a bounds-only header decode β so a 100 MP image does not allocate a full bitmap.Changelog
[GENERAL] [BREAKING] -
getImageMetaDatarenamesImageWidth/ImageHeight/Orientationtowidth/height/orientationand removes the rawexifpassthrough; see the README migration table[GENERAL] [BREAKING] -
getVideoMetaDataremoves the undeclared, Android-onlycreationTimekey[GENERAL] [ADDED] -
getVideoMetaDatareturnsdisplayWidth/displayHeight/rotation,hasVideo/hasAudio,isHDR/colorTransfer,codec,bitrate,frameRate,trackCount,fileName, parsedlatitude/longitude, and the audio track's bitrate/sample rate/channels[GENERAL] [ADDED] -
getImageMetaDatareturnsdisplayWidth/displayHeight/rotation/isMirrored,mimeType,hasAlpha,fileName,isAnimated/frameCount,bitDepth,colorSpace,dpiX/dpiY, signed GPS coordinates, and the camera fields (make,model,lensModel,software,iso,exposureTime,focalLength,flash)[GENERAL] [ADDED] - Export the
VideoMetaData,ImageMetaDataandColorTransferresponse types from the package root[IOS] [FIXED] -
getVideoMetaDatano longer hangs forever on audio-only files or paths not starting withfile://; every path now resolves or rejects[IOS] [FIXED] - Booleans no longer reach JS as numbers β
HybridCompressorunwrapsCFBooleanbefore handing values to Nitro, which affects everyAnyMapresult includingupload[ANDROID] [FIXED] -
getImageMetaDatano longer returnsNaNdimensions for images without EXIF, and no longer rejects when the EXIF block is missing[ANDROID] [FIXED] -
getVideoMetaDatano longer throws a null-pointer exception on audio-only files[ANDROID] [FIXED] -
getVideoMetaDatano longer leaks itsMediaMetadataRetriever[ANDROID] [CHANGED] - Image metadata now uses
androidx.exifinterfaceinstead of the deprecated framework reader, fixing HEIC/WebP/PNG/DNG metadata on many devices[GENERAL] [CHANGED] - The bare example's Image and Video screens display the full metadata for both the source and the compressed output
[INTERNAL] [CHANGED] -
harnessand__tests__are now covered byyarn typecheck, and excluded from the published declarationsTest Plan
yarn test:pryarn test:harness:androidoryarn test:harness:ios.yarn test:prβ exit 0yarn test:harness:iosβ 9/9 passed on a booted iPhone 17 Pro simulatorThe harness assertions were extended to cover the new contract:
rotationβ{0, 90, 180, 270}, display dimensions equal the stored ones swapped only on a quarter turn, andisHDRagrees withcolorTransfer. Actual response for the video fixture:{ "width": 320, "height": 240, "displayWidth": 320, "displayHeight": 240, "rotation": 0, "hasVideo": true, "hasAudio": true, "isHDR": false, "colorTransfer": "sdr", "codec": "h264", "bitrate": 67144, "frameRate": 30, "trackCount": 2, "audioBitrate": 96906, "audioSampleRate": 44100, "audioChannels": 1, "duration": 2, "size": 44420, "extension": "mp4", "fileName": "compressor-harness-video.mp4" }This is what caught the
hasVideo: 1boolean bug β the type mismatch was invisible to the unit tests, which mock the native module entirely.Native compile checks β both clean, with no new warnings in any touched native file:
Bare example on a physical iPhone β built, installed and launched:
Not verified
yarn test:harness:androidwas not run. The APK builds and installs cannot proceed on the available device: a MIUI phone rejects every install path withINSTALL_FAILED_USER_RESTRICTED(gradle installDebug,adb install, andadb push+adb shell pm installall fail identically, since MIUI enforces this inside PackageManager), and the harness's emulator path cannot run while any physical device is attached because it callsemu avd nameon every attached device.So the Android side of this change is compile-verified only. The parts that most deserve a second pair of eyes, or a run on a working emulator:
ImageMain.kt(a hand-written container walk β the newest, least-proven code here);NaN-dimensions fix via the bounds-onlyBitmapFactoryread;parseIso6709and the AndroidX EXIF GPS conversion for southern/western coordinates, where a sign error cannot be caught by a northern-hemisphere fixture.