How to Handle HDR Source Videos with PQ or HLG Transfer Functions in video-use

The video-use library automatically detects HDR sources using PQ (SMPTE 2084) or HLG (ARIB STD-B67) transfer functions and applies a dedicated tone-mapping filter chain to convert them to Rec. 709 SDR without blown-out colors.

Working with modern HDR footage from smartphones or professional cameras requires careful handling of transfer functions. The browser-use/video-use repository implements automatic HDR detection and tone mapping to handle HDR source videos with PQ or HLG transfer functions seamlessly. This ensures that HDR10 and HLG clips are correctly converted to standard SDR output during the rendering pipeline.

HDR Detection via ffprobe in helpers/render.py

The detection logic resides in helpers/render.py within the is_hdr_source() function. This utility runs an ffprobe command to inspect the color_transfer metadata of the first video stream.

HDR_TRANSFERS = {"smpte2084", "arib-std-b67"}   # PQ (HDR10) and HLG

When is_hdr_source() returns True, the system recognizes the source as HDR and triggers the tone-mapping pipeline. The smpte2084 value indicates PQ (Perceptual Quantizer) used in HDR10, while arib-std-b67 indicates Hybrid Log-Gamma (HLG).

The Tone Mapping Chain Implementation

For HDR sources, video-use constructs a specific FFmpeg video filter (vf) string defined as TONEMAP_CHAIN in helpers/render.py:

TONEMAP_CHAIN = (
    "zscale=t=linear:npl=100,"
    "format=gbrpf32le,"
    "zscale=p=bt709,"
    "tonemap=tonemap=hable:desat=0,"
    "zscale=t=bt709:m=bt709:r=tv,"
    "format=yuv420p"
)

This chain performs six critical operations:

1. Linear light conversion – zscale=t=linear transforms the source PQ or HLG gamma curve to linear light with 100 nits peak luminance.
2. High-precision format – format=gbrpf32le converts to 32-bit floating point for artifact-free processing.
3. Color space primaries – zscale=p=bt709 maps the wide color gamut to BT.709 primaries.
4. Tone mapping operator – tonemap=tonemap=hable:desat=0 applies the Hable tone mapping curve to compress dynamic range without desaturation.
5. BT.709 output setup – zscale=t=bt709:m=bt709:r=tv sets transfer characteristics, matrix coefficients, and TV range for BT.709.
6. Final formatting – format=yuv420p outputs standard 8-bit 4:2:0 YUV for maximum compatibility.

Integration into Segment Extraction

The tone mapping chain integrates conditionally in the extract_segment() function. When processing video segments, the code checks for HDR sources and prepends the filter chain before any scaling or grading operations:

if is_hdr_source(source):
    vf_parts.append(TONEMAP_CHAIN)
vf_parts.append(scale)               # portrait/landscape scaling

if grade_filter:
    vf_parts.append(grade_filter)    # optional color-grade per segment

vf = ",".join(vf_parts)

This ensures that HDR conversion happens first, allowing subsequent filters like scaling or color grading to work with standard SDR luminance levels. The final -vf argument supplied to FFmpeg contains the tone-mapping steps before any other processing.

Why Proper Tone Mapping Matters

Without this tone mapping chain, converting HDR sources by merely changing bit depth (e.g., yuv420p10le to yuv420p) preserves the original PQ or HLG transfer metadata. Media players that respect this metadata interpret the 8-bit values as HDR, resulting in oversaturated, blown-out images with incorrect gamma curves.

The video-use implementation strips the HDR transfer function, maps luminance correctly using the Hable operator, and writes a standard Rec. 709 container. This produces SDR-safe output suitable for screen recordings, social media uploads, and downstream processing workflows that expect standard dynamic range.

Customizing the Tone Mapping Operator

You can modify the tone mapping behavior by editing the TONEMAP_CHAIN constant in helpers/render.py. For example, to use the Reinhard operator instead of Hable:

TONEMAP_CHAIN = (
    "zscale=t=linear:npl=100,"
    "format=gbrpf32le,"
    "zscale=p=bt709,"
    "tonemap=tonemap=reinhard:desat=0,"
    "zscale=t=bt709:m=bt709:r=tv,"
    "format=yuv420p"
)

After saving changes to helpers/render.py, subsequent renders will apply your custom operator to all detected HDR sources. Available alternatives in FFmpeg include mobius and clip, each offering different trade-offs between highlight preservation and contrast.

Summary

• video-use automatically detects HDR sources in helpers/render.py by checking color_transfer values for smpte2084 (PQ) or arib-std-b67 (HLG) via is_hdr_source().
• The TONEMAP_CHAIN applies a six-step FFmpeg filter to convert HDR to SDR using linear light transformation, Hable tone mapping, and BT.709 color space conversion.
• HDR conversion occurs in extract_segment() before scaling or grading to ensure downstream filters receive normalized SDR data.
• Without tone mapping, HDR metadata causes display issues in standard media players; the built-in chain prevents blown-out colors and saturation errors while maintaining correct luminance.

Frequently Asked Questions

How does video-use detect HDR video sources?

The system runs ffprobe via the is_hdr_source() function in helpers/render.py to read the color_transfer stream metadata. If the value matches smpte2084 (PQ/HDR10) or arib-std-b67 (HLG), the source is flagged as HDR and the tone mapping chain is automatically activated during segment extraction.

Can I use a different tone mapping algorithm besides Hable?

Yes. Edit the TONEMAP_CHAIN constant in helpers/render.py and change tonemap=hable to alternatives like reinhard, mobius, or clip. After saving the file, the new operator applies to all future HDR processing without requiring changes to your workflow or source files.

Why does my HDR video look washed out or oversaturated without tone mapping?

Simple bit-depth conversion preserves HDR transfer metadata while stripping the actual HDR luminance data. Players read the metadata and interpret the 8-bit values as HDR, causing incorrect gamma curves and blown highlights. The tone mapping chain properly converts the transfer function to BT.709 SDR, preventing these display artifacts.

Does the tone mapping affect SDR sources?

No. The TONEMAP_CHAIN only appends to the filter string when is_hdr_source() returns True. SDR sources bypass the HDR processing pipeline entirely and proceed directly to scaling and grading operations in extract_segment().

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}