Instruments the video upload pipeline with a typed event family
(picked, compress{Started,Completed,Skipped,Failed}, upload{Started,Completed,Failed},
processing{Started,Completed,Failed}, published, abandoned) plus a parent
Sentry span with child spans per phase. Every event carries an uploadId,
engine identifier, and jobId once known.
Implements the Metrics workstream from the Video v2 frontend spec (§D):
fleet visibility into per-phase timings, codec paths, file-size
distributions, and abandonment phase. No content is captured — sizes,
codecs, dimensions, timings, and error class names only.
Telemetry is created alongside the abort controller in Composer.tsx and
threaded through the composer reducer onto VideoState so onPressPublish
can fire the published event once apilib.post resolves. The abort signal
drives the abandoned event with the phase the user bailed in.
Per Frontend Spec (A2.4 pre-merge note): MAX_UPLOAD_SIZE was hardcoded to
100 MB and would force-compress 100-300 MB files that should skip. Reads
from VIDEO_MAX_SIZE now. Pulled the target/passthrough bitrate and max
dimension out of compress.ts into src/lib/media/video/constants.ts so the
compression contract lives in one place without bloating the global
constants file.
- Android non-AAC audio is now transcoded to AAC (matching iOS) via a
source-decoder -> AAC-encoder pre-pass that captures the encoder's
output format before the muxer starts, then writes buffered samples
after the video pipeline finishes. Falls back to dropping audio if
the transcode fails.
- compress.ts wraps probe() in try/catch and falls through to
passthrough on failure instead of throwing.
- Probers expose isHDR (HLG/PQ via color transfer, plus Dolby Vision
codecs/mimes); shouldCompress forces compression for HDR sources so
the SDR BT.709 path always runs. Also sets
KEY_COLOR_TRANSFER_REQUEST=SDR on the Android decoder (API 31+) so
HDR sources tone-map to SDR pixels instead of being mislabeled.
N1: Delete unused CodecSelector.selectEncoder. VideoCompressor calls
findEncoder directly; selectEncoder was dead since the initial commit.
N2: Switch Android output filename from System.currentTimeMillis() to
UUID.randomUUID(). Eliminates the collision risk when two compressions
start in the same millisecond.
N3: Guard CheckedContinuation against double-resume in both
processVideoTrack and processAudioTrack. AVAssetWriter shouldn't
re-invoke the requestMediaDataWhenReady block after markAsFinished, but
if it did, the previous code would crash via CheckedContinuation's
double-resume detection. Added a `finished` flag plus a local `finish`
closure that no-ops on second call.
N4: DataRateLimits window duration changed from Int `1` to Double `1.0`.
VideoToolbox expects the duration value as a CFNumber with float
semantics; Int bridges to NSNumber(int) which works on most iOS
versions but is not spec-correct.
B1: Android encoder format now sets KEY_COLOR_STANDARD = BT709,
KEY_COLOR_TRANSFER = SDR_VIDEO, KEY_COLOR_RANGE = LIMITED (API 24+).
Previously the encoder inherited or emitted default color metadata,
which meant HDR sources produced incorrectly-tagged output on Android.
iOS already had AVVideoColorPropertiesKey set correctly.
B2: Replace single currentCompressor reference with a per-job map
keyed by jobId. The cancel function now takes a jobId and only cancels
that specific job. Prevents the previous race where a second compress
call overwrote the reference and made the first job uncancellable.
B3: Clamp frameRateCap to >= 1 at the native module boundary. Previously
a value of 0 from JS would cause divide-by-zero (Android Long division
ArithmeticException, iOS CMTime Infinity / Int32(0) frameDuration trap).
Replaces react-native-compressor's video path with a local Expo module.
Native pipeline:
- iOS: AVAssetReader + AVMutableVideoComposition + AVAssetWriter with
VideoToolbox encode, BGRA reader for HDR -> SDR conversion, BT.709
color tagging, DataRateLimits hard cap, AAC re-encode.
- Android: MediaExtractor -> MediaCodec(decoder) -> GL pipeline ->
MediaCodec(encoder) -> MediaMuxer. BITRATE_MODE_CBR for tight target
enforcement, GL transform-matrix rotation (no double-rotation),
raw AAC passthrough, hardware encoder selection with software
fallback, QTI AVC denylist.
Codec selection: 'auto' resolves to h264 (HLS pipeline + licensing).
HEVC remains opt-in via codec: 'hevc' for future feature-flagged use.
compress.ts adds probe-based smart-skip: clips that are already at or
below 5 Mbps / 1920px / 100MB bypass re-encoding entirely.
react-native-compressor stays as a dep for now since pickVideo and
VideoTranscodeBackdrop still use its non-compression helpers; full
removal is a follow-up.
vineyardbovines