How video-use Handles HDR (HLG/PQ) Source Footage from iPhones and Professional Cameras

video-use automatically detects HDR content using ffprobe and applies a zscale-based tone-mapping filter chain to convert HLG and PQ footage to standard SDR before rendering.

When processing footage from iPhones or cinema cameras that record in HLG or PQ (HDR10), the browser-use/video-use repository ensures proper color reproduction by automatically tone-mapping high dynamic range content to standard dynamic range. This eliminates blown-out highlights and oversaturated colors that occur when HDR footage is incorrectly processed as SDR.

HDR Detection via ffprobe in helpers/render.py

The detection logic centers on the is_hdr_source function in helpers/render.py, which inspects video stream metadata to identify HDR transfer functions.

When processing a clip, the function executes an ffprobe command that extracts the color_transfer tag:

out = subprocess.run(
    ["ffprobe", "-v", "error", "-select_streams", "v:0",
     "-show_entries", "stream=color_transfer",
     "-of", "default=noprint_wrappers=1:nokey=1", str(video)],
    capture_output=True, text=True, check=True,
)
return out.stdout.strip() in HDR_TRANSFERS

The HDR_TRANSFERS set is defined as {"smpte2084", "arib-std-b67"}, corresponding to PQ (HDR10) and HLG respectively. This allows the pipeline to distinguish between standard SDR footage and HDR content that requires tone-mapping.

The TONEMAP_CHAIN Filter Pipeline

Upon detecting HDR content, video-use prepends a predefined filter graph called TONEMAP_CHAIN to the FFmpeg video filter string. This chain converts HDR colors in the Rec.2020 space to SDR Rec.709 using the Hable tone-mapping operator.

The filter string defined in helpers/render.py (lines 108-118) executes the following transformations:

TONEMAP_CHAIN = (
    "zscale=t=linear:npl=100,"
    "format=gbrpf32le,"
    "zscale=p=bt709,"
    "tonemap=tonemap=hable:desat=0,"
    "zscale=t=bt709:m=bt709:r=tv,"
    "format=yuv420p"
)

This sequence first converts to linear light, then applies the Hable tone-mapping curve with zero desaturation, followed by BT.709 primaries and matrix conversion. The result is a properly exposed SDR image that preserves highlight detail without the oversaturated appearance typical of unconverted HDR.

Integration in the Extraction Pipeline

The extract_segment function integrates HDR handling seamlessly into the rendering workflow. Located in helpers/render.py (lines 179-186), this function constructs the video filter list by conditionally inserting TONEMAP_CHAIN as the first element when is_hdr_source returns True.

The filter construction follows this order:

1. TONEMAP_CHAIN (if HDR detected)
2. Scaling filters (e.g., to 1080p)
3. Color grade filters

This ensures tone-mapping occurs before any spatial transformations or color grading, maintaining color accuracy throughout the pipeline. The final filter string is passed to FFmpeg via the -vf argument, guaranteeing consistent output regardless of whether the source is iPhone HLG footage or professional PQ HDR content.

Practical Usage Examples

Processing an EDL containing iPhone HDR footage requires no special flags—the detection and conversion happen automatically:

# iPhone footage with HLG color transfer is auto-detected

python helpers/render.py edl.json -o final.mp4

The render.py script executes the full pipeline: detecting the HLG transfer function via arib-std-b67, prepending the TONEMAP_CHAIN filter, and outputting SDR-compatible video.

For programmatic access to the tone-mapping pipeline:

from helpers.render import extract_segment
from pathlib import Path

extract_segment(
    source=Path("iphone_hlg_clip.mov"),
    seg_start=0,
    duration=15,
    grade_filter="",
    out_path=Path("output_sdr.mp4"),
)

The function automatically identifies HDR sources and applies the necessary conversion without manual intervention.

Summary

• video-use detects HDR content by inspecting color_transfer metadata using ffprobe in helpers/render.py.
• The is_hdr_source function identifies PQ (smpte2084) and HLG (arib-std-b67) transfer functions.
• Detected HDR clips trigger the TONEMAP_CHAIN filter, which uses zscale and Hable tone-mapping to convert Rec.2020 HDR to Rec.709 SDR.
• The conversion happens automatically in extract_segment before scaling or grading, ensuring consistent color reproduction across iPhone and professional camera footage.

Frequently Asked Questions

Does video-use support both HLG and PQ HDR formats?

Yes. The is_hdr_source function in helpers/render.py explicitly checks for both arib-std-b67 (HLG, used by iPhones) and smpte2084 (PQ/HDR10, used by professional cameras). Both formats trigger the same TONEMAP_CHAIN filter pipeline for conversion to SDR.

Why does video-use use the Hable tone-mapping operator instead of other methods?

The TONEMAP_CHAIN utilizes the Hable operator (tonemap=hable:desat=0) because it provides a smooth roll-off of highlights while preserving color saturation. The desat=0 parameter specifically maintains color intensity during the conversion from HDR to SDR, preventing the washed-out appearance common with other tone-mapping curves.

Will tone-mapping affect already-processed SDR footage?

No. The HDR detection logic only applies the TONEMAP_CHAIN when is_hdr_source returns True. Standard SDR footage with BT.709 color transfer characteristics will bypass the tone-mapping stage entirely, proceeding directly to scaling and grading filters without additional processing overhead.

Can I customize the tone-mapping parameters for specific projects?

Currently, the TONEMAP_CHAIN is defined as a constant in helpers/render.py (lines 108-118). To modify the tone-mapping behavior, you would need to edit this constant directly in the source code, adjusting parameters such as the npl (nominal peak luminance) value or switching to a different tone-mapping algorithm like reinhard or mobius.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}