Fragmented MP4 (fMP4) Recording Implementation in FadCam: Complete Technical Guide

FadCam implements fragmented MP4 recording through FragmentedMp4MuxerWrapper, which wraps AndroidX Media3's FragmentedMp4Muxer to generate 2-second streamable fragments with per-track timestamp normalization, enabling crash-safe local storage and real-time HLS streaming without file corruption.

FadCam is an open-source Android camera application that records video using fragmented MP4 (fMP4)—a container format that writes moov atom metadata incrementally rather than deferring it to the end of the file. This architecture makes recorded files instantly playable and resistant to corruption if the app crashes or the device loses power. According to the FadCam source code in anonfaded/FadCam, the implementation centers on adapting Android's MediaCodec output to Media3's muxing pipeline while handling complex timestamp normalization and live-streaming callbacks.

Core Architecture: FragmentedMp4MuxerWrapper

The recording pipeline centers on FragmentedMp4MuxerWrapper, which acts as an adapter between Android's MediaCodec encoder and Media3's fragmented MP4 muxer. This wrapper solves three critical problems: timestamp normalization, format conversion, and fragment-based streaming.

Media3 Muxer Integration

The wrapper instantiates FragmentedMp4Muxer with a callback consumer that receives each completed fragment. The muxer emits init segments (containing ftyp and moov atoms) and media fragments (containing moof and mdat atoms) every 2 seconds.

Consumer<ProcessedSegment> segmentConsumer = segment -> handleProcessedSegment(segment);
this.muxer = new FragmentedMp4Muxer.Builder(segmentConsumer)
        .setFragmentDurationMs(2000)          // 2-second fragments
        .build();

The 2-second fragment duration is explicitly tuned for stable live-streaming and seeking behavior, as noted in the source comments regarding "CRITICAL FIX for fMP4 seeking."

Track Registration and Format Conversion

Tracks are registered via addTrack(MediaFormat), which converts Android's MediaFormat to Media3's Format object. The wrapper extracts and preserves:

• Video parameters: width, height, frame-rate, bit-rate, and color information (BT.709/BT.2020 color spaces) required for proper VLC playback of HEVC content.
• Audio parameters: Sample rate, channel count, and a cleanly generated AAC AudioSpecificConfig for proper esds box creation (mp4a.40.2).

Track indices (videoTrackIndex, audioTrackIndex) are stored to handle flag conversion differently per track—ensuring AAC samples are never incorrectly marked as key frames.

Timestamp Normalization Strategy

Android MediaCodec timestamps are system-uptime based and monotonically increasing across recording sessions. Without normalization, a new recording would incorrectly appear as a 45-minute-old video. FadCam solves this through per-track timestamp offset mapping.

Offset Calculation Logic

The wrapper maintains a SparseArray<Long> named timestampOffsets. When the first sample arrives for each track, the wrapper captures its presentation timestamp (PTS) as the baseline offset. All subsequent samples subtract this offset, forcing timestamps to start at 0 microseconds for every new recording.

if (!timestampOffsetsInitialized) {
    if (timestampOffsets.get(trackIndex) == null) {
        timestampOffsets.put(trackIndex, bufferInfo.presentationTimeUs);
    }
    if (timestampOffsets.size() >= trackCount) {
        timestampOffsetsInitialized = true;
    }
}
long normalizedPts = bufferInfo.presentationTimeUs - timestampOffsets.get(trackIndex);

This normalization occurs within writeSampleData before forwarding buffers to the Media3 muxer.

Sample Processing and Fragment Emission

The writeSampleData method receives raw ByteBuffer objects from the GL recording pipeline. The wrapper validates that buffer.remaining() matches the reported size but does not re-position the buffer, maintaining zero-copy efficiency where possible.

Buffer Flags and Media3 Conversion

The wrapper constructs Media3 BufferInfo objects using the normalized PTS and converts Android flags to Media3 constants:

• MediaCodec.BUFFER_FLAG_KEY_FRAME → C.BUFFER_FLAG_KEY_FRAME
• MediaCodec.BUFFER_FLAG_END_OF_STREAM → C.BUFFER_FLAG_END_OF_STREAM

Fragment Callback Handling

The handleProcessedSegment method processes two fragment types:

1. Init segments (segment.isInitSegment): Contains file type and movie header atoms. These are sent to RemoteStreamManager and, when in STREAM_AND_SAVE mode, written to the local file descriptor.
2. Media fragments: Contains movie fragment boxes and media data. These are uploaded to the HLS server via RemoteStreamManager.onFragmentComplete and optionally persisted to disk.

The callback validates the underlying FileDescriptor before writing, preventing EBADF errors during abrupt shutdowns.

Metadata and Configuration

Orientation and Timestamps

Before calling start(), the wrapper injects orientation metadata using Mp4OrientationData to support video rotation hints (e.g., 90° for portrait). Creation timestamps are converted from Unix epoch to MP4 epoch (1904-01-01) using Mp4TimestampData.unixTimeToMp4TimeSeconds, ensuring compatibility with standard MP4 parsers.

Color Information Extraction

For video tracks, the wrapper extracts color space, range, and transfer characteristics from MediaFormat and populates a Media3 ColorInfo object. This metadata is essential for VLC and other players to correctly render HDR or wide-gamut content without color distortion.

Live Streaming Integration

FadCam's fMP4 implementation supports simultaneous local recording and remote streaming. The FragmentedMp4MuxerWrapper communicates with RemoteStreamManager through the ProcessedSegment callback interface.

When streaming is active:

• Init segments initialize the HLS stream manifest
• Media fragments are uploaded as discrete files for adaptive bitrate streaming
• The wrapper ensures fragments are written to disk only when STREAM_AND_SAVE mode is active, optimizing I/O for pure streaming scenarios

Implementation Example

The following pattern demonstrates typical usage within FadCam's recording pipeline:

// Initialize wrapper with output path
FragmentedMp4MuxerWrapper muxer = new FragmentedMp4MuxerWrapper("/sdcard/Movies/fadcam.mp4");

// Register tracks from MediaCodec formats
int videoTrack = muxer.addTrack(videoMediaFormat);
int audioTrack = muxer.addTrack(audioMediaFormat);

// Set metadata
muxer.setOrientationHint(90);
muxer.setLocation(37.7749f, -122.4194f);

// Begin muxing
muxer.start();

// During encoding (called from GLRecordingPipeline)
muxer.writeSampleData(videoTrack, encodedVideoBuffer, bufferInfo);
muxer.writeSampleData(audioTrack, encodedAudioBuffer, bufferInfo);

// Finalize
muxer.stop();    // Writes final fragment
muxer.release(); // Closes FileDescriptor and persists moov

Key Source Files and Responsibilities

File	Role
app/src/main/java/com/fadcam/media/FragmentedMp4MuxerWrapper.java	Core wrapper adapting MediaCodec to Media3's FragmentedMp4Muxer; handles timestamp normalization, format conversion, and live-stream callbacks.
app/src/main/java/com/fadcam/streaming/RemoteStreamManager.java	Receives fragment callbacks and manages HLS upload; handles init segments and media fragment dispatch.
app/src/main/java/com/fadcam/playback/FragmentedMp4Remuxer.java	Utility for re-packaging existing MP4 files into fragmented format for playback compatibility.
app/src/main/java/com/fadcam/playback/FragmentedMp4IndexBuilder.java	Constructs seek tables for fragmented MP4 files, enabling random access during ExoPlayer playback.
app/src/main/java/com/fadcam/playback/SeekableFragmentedMp4MediaSourceFactory.java	Factory creating MediaSource instances capable of seeking within fMP4 containers for PlaybackManager integration.

Summary

• FragmentedMp4MuxerWrapper serves as the bridge between Android's MediaCodec and Media3's fragmented MP4 muxer, handling format conversion and timestamp normalization.
• 2-second fragment duration balances streaming latency with encoding efficiency, emitting moof/mdat pairs via the ProcessedSegment callback.
• Per-track timestamp offsets reset presentation timestamps to zero for each recording, preventing "old video" artifacts caused by system uptime-based clock sources.
• Dual-path output supports both local file persistence and real-time HLS streaming through RemoteStreamManager, with validation to prevent write errors on invalid file descriptors.
• Color and AAC metadata extraction ensures compatibility with VLC and standard media players, preserving HDR color spaces and proper audio codec configuration.

Frequently Asked Questions

What makes fragmented MP4 different from standard MP4 recording?

Standard MP4 writes the moov atom (metadata index) at the end of the file, requiring the entire recording to finish before the file becomes playable. Fragmented MP4 writes the movie box incrementally alongside moof (movie fragment) headers, making each 2-second segment independently playable and streamable. In FadCam, this means recorded files survive app crashes and can be streamed to remote servers while still recording.

How does FadCam handle timestamp issues during recording?

FadCam solves the system uptime timestamp problem by maintaining a SparseArray<Long> of offsets per track. The first sample's PTS becomes the baseline for that track, and the wrapper subtracts this offset from all subsequent samples. This normalization forces every recording to start at 0 microseconds regardless of how long the device has been running.

Why does FadCam use 2-second fragments specifically?

The 2-second fragment duration (configured via setFragmentDurationMs(2000)) represents a balance between live-streaming latency and encoding overhead. Shorter fragments reduce latency for HLS streaming but increase file size due to repeated moof headers. The 2-second value is explicitly tuned for seeking stability, as noted in the FadCam source code comments regarding the critical fix for fMP4 seeking behavior.

Can FadCam record and stream simultaneously?

Yes. When configured for STREAM_AND_SAVE mode, FragmentedMp4MuxerWrapper sends init segments and media fragments to both the local file descriptor and RemoteStreamManager. The handleProcessedSegment callback validates the FileDescriptor before disk writes while simultaneously uploading fragments to the HLS server, enabling real-time broadcasting without sacrificing local recording quality.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}