How HDR to SDR Tone Mapping Works with zscale and Hable Desaturation in video-use

The video-use repository automatically converts HDR10 and HLG sources to SDR using a FFmpeg filter chain that linearizes light with zscale, applies the Hable filmic tonemap with desaturation disabled, and outputs standard BT.709 8-bit video.

The browser-use/video-use project handles HDR to SDR tone mapping internally when processing video segments, eliminating the need for manual color space conversion. This automatic pipeline ensures that high dynamic range footage captured in PQ (HDR10) or HLG formats can be distributed as standard MP4 files compatible with SDR displays and social media platforms.

Detecting HDR Sources

HDR detection occurs in helpers/render.py within the is_hdr_source() function (lines 120–130). The utility probes the video stream using ffprobe and checks the color_transfer property against a predefined set of HDR transfer functions:

HDR_TRANSFERS = {"smpte2084", "arib-std-b67"}   # PQ (HDR10) and HLG

When extract_segment() processes a video file, it invokes is_hdr_source(source) to determine whether the incoming footage requires tone mapping. If the transfer function matches either SMPTE 2084 (PQ) or ARIB STD-B67 (HLG), the pipeline flags the source as HDR and prepares the conversion chain.

The zscale and Hable Tone Mapping Pipeline

The core conversion logic resides in the TONEMAP_CHAIN constant defined in helpers/render.py (lines 111–116). This string constructs a six-stage FFmpeg filter graph that transforms HDR linear light into display-referred SDR video.

Linearizing Light with zscale

The chain begins by converting the non-linear HDR signal to linear light intensity:

zscale=t=linear:npl=100

The t=linear parameter removes the gamma/transfer function, while npl=100 normalizes peak luminance to 100 nits. This normalization aligns the source data with the Hable operator's reference assumptions.

Precision and Color Space Conversion

Next, the pipeline switches to high-precision floating-point format and converts to BT.709 primaries:

format=gbrpf32le,zscale=p=bt709

The gbrpf32le format prevents banding during the tonemapping operation, and p=bt709 re-interprets the linear data using the standard SDR color primaries.

Applying the Hable Tonemap

The central tone mapping operation uses the filmic Hable curve:

tonemap=tonemap=hable:desat=0

Hable is a filmic tonemapping operator derived from Unreal Engine's color grading pipeline that compresses high dynamic range highlights while preserving shadow detail. The desat=0 parameter explicitly disables the default color desaturation behavior, ensuring that chroma remains fully saturated rather than shifting hues to avoid artifacts.

Final Output Formatting

The chain concludes by converting back to non-linear gamma and limiting to TV range:

zscale=t=bt709:m=bt709:r=tv,format=yuv420p

This applies BT.709 transfer characteristics (t=bt709), sets the color matrix (m=bt709), clamps levels to 16-235 (r=tv), and outputs to the widely-supported 8-bit YUV420 pixel format.

The complete constant as implemented in the source:

TONEMAP_CHAIN = (
    "zscale=t=linear:npl=100,"
    "format=gbrpf32le,"
    "zscale=p=bt709,"
    "tonemap=tonemap=hable:desat=0,"
    "zscale=t=bt709:m=bt709:r=tv,"
    "format=yuv420p"
)

Integrating the Pipeline into Segment Extraction

Inside extract_segment() (near line 198), the script dynamically constructs the video filter (vf) graph by conditionally prepending the tone mapping chain:

vf_parts: list[str] = []
if is_hdr_source(source):
    vf_parts.append(TONEMAP_CHAIN)   # HDR → SDR conversion

vf_parts.append(scale)               # 1080p/720p scaling

if grade_filter:
    vf_parts.append(grade_filter)
vf = ",".join(vf_parts)

This ensures that HDR conversion occurs before resolution scaling and color grading filters are applied. The final FFmpeg command incorporates the complete filter string via -vf <vf>, performing the tone map during the encoding pass.

Why zscale and Hable?

zscale provides broadcast-quality color space conversion through the zimg library, offering full-precision arithmetic and explicit control over transfer functions that standard FFmpeg scale filters lack. Using zscale twice—first to linearize and later to apply BT.709 gamma—guarantees mathematically correct HDR-to-SDR transformation.

Hable produces a pleasing, contrast-preserving roll-off that avoids the hard-clipping artifacts common in simple gamma-based conversions. By setting desat=0, the pipeline preserves the original color saturation, which is typically preferred for social media content over the hue-protecting desaturation that the default Hable implementation applies.

Practical Implementation Examples

Inspecting the tone mapping configuration programmatically:

from helpers.render import TONEMAP_CHAIN, is_hdr_source
from pathlib import Path

print("Filter chain:", TONEMAP_CHAIN)
print("Is HDR?", is_hdr_source(Path("input_hdr.mov")))

Applying the same conversion manually with FFmpeg:

ffmpeg -i input_hdr.mov -vf \
"zscale=t=linear:npl=100,format=gbrpf32le,\
zscale=p=bt709,tonemap=tonemap=hable:desat=0,\
zscale=t=bt709:m=bt709:r=tv,format=yuv420p" \
-c:v libx264 -crf 20 -c:a copy output_sdr.mp4

Summary

• HDR Detection: The is_hdr_source() function in helpers/render.py identifies PQ and HLG content by checking for smpte2084 or arib-std-b67 transfer functions.
• Linear Conversion: The TONEMAP_CHAIN uses zscale=t=linear:npl=100 to normalize HDR luminance to 100 nits in linear light space.
• Filmic Mapping: The Hable operator compresses dynamic range with desat=0 to preserve color saturation rather than protecting hues through desaturation.
• Output Standardization: The chain outputs to BT.709 with TV-range levels and YUV420p format for universal compatibility.
• Pipeline Integration: Tone mapping is prepended to the filter graph in extract_segment() before scaling or grading operations are applied.

Frequently Asked Questions

What HDR formats does video-use support?

The pipeline supports HDR10 (PQ/SMPTE 2084) and HLG (ARIB STD-B67) encoded sources. These are detected via the color_transfer metadata field using the HDR_TRANSFERS set defined in helpers/render.py.

Why does the Hable tonemap use desat=0?

Setting desat=0 disables the Hable operator's default color desaturation behavior. While the default desaturation protects against hue shifts in extreme highlights, disabling it preserves the original color saturation, which is generally preferred for content destined for social media platforms.

Can I modify the tone mapping parameters?

Currently, the TONEMAP_CHAIN is defined as a constant in helpers/render.py (lines 111–116). To adjust parameters such as peak luminance (npl) or tonemap operators, you would need to modify this constant directly in the source code before rendering.

Where in the code is the tone mapping actually applied?

The tone mapping filter chain is applied in helpers/render.py inside the extract_segment() function. When is_hdr_source() returns True, the TONEMAP_CHAIN string is inserted at the beginning of the vf_parts list, ensuring it processes the video before scaling or color grading filters are appended.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}