Complete Guide to FFmpeg Filter Chains in video-use: HDR Tone Mapping, Color Grading, and Compositing

The video-use repository constructs complex FFmpeg pipelines using modular filter chains for HDR-to-SDR conversion, dynamic color grading, resolution scaling, and multi-layer compositing, primarily implemented in helpers/render.py and helpers/grade.py.

The browser-use/video-use project processes video through sophisticated FFmpeg command generation. Understanding the specific ffmpeg filter chains utilized throughout the codebase reveals how the tool handles professional video workflows—from HDR tone mapping to automated color correction and loudness normalization.

HDR to SDR Tone Mapping

When the source contains PQ (Perceptual Quantizer) or HLG (Hybrid Log-Gamma) content, the system prepends a dedicated tonemapping chain. In helpers/render.py (lines 110–117), the TONEMAP_CHAIN constant defines a zscale-based pipeline:

TONEMAP_CHAIN = (
    "zscale=t=linear:npl=100,format=gbrpf32le,"
    "zscale=p=bt709,tonemap=tonemap=hable:desat=0,"
    "zscale=t=bt709:m=bt709:r=tv,format=yuv420p"
)

This filter chain linearizes the input, converts to 32-bit floating point for precision, applies hable tonemapping to the BT.709 color space, and converts back to 8-bit YUV420P for delivery.

Resolution Scaling Strategies

The repository adapts output resolution based on quality presets. In helpers/render.py (lines 173–179), the scaling logic selects between portrait and landscape orientations:

• Final output: scale=-2:1920 (portrait) or scale=1920:-2 (landscape)
• Preview mode: scale=-2:1280 or scale=1280:-2
• Draft mode: scale=-2:720 or scale=720:-2

The -2 value preserves aspect ratio by automatically calculating the converse dimension while ensuring the value is divisible by 2 for codec compatibility.

Color Grading and Correction

Preset-Based Color Grades

The helpers/grade.py file (lines 38–60) defines three distinct color-grade presets using the eq, colorbalance, and curves filters:

• subtle: eq=contrast=1.03:saturation=0.98
• neutral_punch: eq=contrast=1.06:brightness=0.0:saturation=1.0,curves=master='0/0 0.25/0.23 0.75/0.77 1/1'
• warm_cinematic: eq=contrast=1.12:brightness=-0.02:saturation=0.88,colorbalance=rs=0.02:gs=0.0:bs=-0.03:rm=0.04:gm=0.01:bm=-0.02:rh=0.08:gh=0.02:bh=-0.05,curves=master='0/0 0.25/0.22 0.75/0.78 1/1'

Dynamic Auto-Grade Algorithm

For automated correction, the auto_grade_for_clip() function in helpers/grade.py (lines 250–264) dynamically constructs eq= filter chains. The system analyzes clip statistics and emits only adjusted parameters, producing strings like:

eq=contrast=1.045:gamma=1.032:saturation=0.997

Audio Processing Filters

Fade In/Out Handling

To prevent audio popping at clip boundaries, helpers/render.py (lines 88–90) applies an afade filter chain:

af = f"afade=t=in:st=0:d=0.03,afade=t=out:st={duration-0.03}:d=0.03"

This creates 30-millisecond fades at both edges of each segment.

Loudness Normalization

The repository implements ITU-R BS.1770-4 loudness normalization using the loudnorm filter. In helpers/render.py (lines 398–470), the measure_loudness() and apply_loudnorm_two_pass() functions generate two variants:

• One-pass: loudnorm=I=-14:TP=-1:LRA=11
• Two-pass: loudnorm=I=-14:TP=-1:LRA=11:measured_I=…:measured_TP=…:measured_LRA=…:measured_thresh=…:offset=…:linear=true

Multi-Layer Compositing and Subtitles

Overlay Positioning and Timing

For picture-in-picture or watermark compositing, the system uses setpts to synchronize overlay timestamps. In helpers/render.py (lines 221–225), each overlay stream receives:

setpts=PTS-STARTPTS+{t}/TB

Where {t} represents the target start time in seconds.

Overlay Compositing Chain

The compositing logic (lines 232–336) chains multiple overlay filters sequentially:

[f"[base][a1]overlay=enable='between(t,{t},{end})'[v1]"]

For multiple overlays, the filter graph builds intermediate labels like [v1], [v2], etc., feeding each subsequent overlay into the previous result.

Subtitle Rendering

The final filter stage applies styled subtitles using the subtitles filter with forced styling. Defined in helpers/render.py (lines 50–56 and 390–404), the SUB_FORCE_STYLE constant ensures consistent typography:

subtitles='path/to/file.srt':force_style='FontName=Helvetica,FontSize=18,Bold=1,PrimaryColour=&H00FFFFFF,OutlineColour=&H00000000,BackColour=&H00000000,BorderStyle=1,Outline=2,Shadow=0,Alignment=2,MarginV=90'

Practical Implementation Examples

Apply a Named Color Grade

To process a file with the warm_cinematic preset:

python helpers/grade.py input.mp4 -o output.mp4 --preset warm_cinematic

This executes an FFmpeg command combining the preset's eq, colorbalance, and curves filters:

ffmpeg -y -i input.mp4 -vf "eq=contrast=1.12:brightness=-0.02:saturation=0.88,colorbalance=rs=0.02:gs=0.0:bs=-0.03:rm=0.04:gm=0.01:bm=-0.02:rh=0.08:gh=0.02:bh=-0.05,curves=master='0/0 0.25/0.22 0.75/0.78 1/1'" -c:v libx264 -preset fast -crf 18 -pix_fmt yuv420p -c:a copy -movflags +faststart output.mp4

Render a Complex Timeline with Overlays

For a final output with multiple overlays and subtitles:

python helpers/render.py edl.json -o final.mp4

The generated filter graph chains scaling, tonemapping (if needed), grading, and overlay compositing:

[0:v]setpts=PTS-STARTPTS+2.500/TB[a1],
[1:v]setpts=PTS-STARTPTS+5.000/TB[a2],
[0:v][a1]overlay=enable='between(t,2.500,7.500)'[v1],
[v1][a2]overlay=enable='between(t,5.000,10.000)'[v2],
[v2]subtitles='subtitles.srt':force_style='FontName=Helvetica,FontSize=18,...'[outv]

Summary

• HDR processing uses a zscale-based TONEMAP_CHAIN in helpers/render.py (lines 110–117) to convert PQ/HLG sources to BT.709 SDR.
• Resolution scaling adapts to orientation and quality mode using scale=-2:1920 (portrait) or scale=1920:-2 (landscape) in helpers/render.py (lines 173–179).
• Color grading offers three presets (subtle, neutral_punch, warm_cinematic) in helpers/grade.py (lines 38–60) plus dynamic auto-grade generation via auto_grade_for_clip() (lines 250–264).
• Audio processing combines 30ms afade filters at clip boundaries and loudnorm for broadcast-standard loudness normalization in helpers/render.py (lines 88–90 and 398–470).
• Compositing chains setpts timing adjustments with sequential overlay filters and styled subtitles for final output delivery.

Frequently Asked Questions

How does video-use handle HDR content that needs SDR conversion?

The repository detects HDR sources and automatically prepends the TONEMAP_CHAIN filter string found in helpers/render.py (lines 110–117). This chain uses zscale for color space conversion, format=gbrpf32le for 32-bit processing precision, and tonemap=hable for smooth roll-off of highlight details, ensuring consistent SDR output without clipping.

Can I customize the color grade presets beyond the three built-in options?

Yes. The PRESETS dictionary in helpers/grade.py (lines 38–60) contains standard FFmpeg eq, colorbalance, and curves filter syntax. You can add new entries following the existing format, or use the auto_grade_for_clip() function (lines 250–264) as a template to generate dynamic eq= chains based on clip-specific luminance and saturation analysis.

Why does the audio processing use 30-millisecond fades specifically?

The 30ms fade duration ( implemented as afade=t=in:st=0:d=0.03 in helpers/render.py, lines 88–90) represents a compromise between perceptual transparency and editing precision. This duration is short enough to avoid audible gaps in dialogue while eliminating click and pop artifacts that occur when cutting audio at non-zero crossings.

What is the purpose of the setpts=PTS-STARTPTS+{t}/TB filter in overlay processing?

This filter expression in helpers/render.py (lines 221–225) synchronizes overlay streams to the main timeline. PTS-STARTPTS resets the overlay's presentation timestamp to zero, while +{t}/TB adds the target offset in timebase units. This ensures overlays appear at precise timestamps regardless of when the source clip begins in the original file.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}