How Auto-Grade Uses ffmpeg signalstats for Brightness and Saturation Analysis

The auto-grade feature in browser-use/video-use analyzes video clips by sampling frames with ffmpeg's signalstats filter to extract per-frame luma and saturation metrics, then applies heuristic thresholds to generate a corrective eq= filter that adjusts contrast, gamma, and saturation within ±8%.

The auto-grade functionality lives in helpers/grade.py and provides an automated way to correct common exposure and color issues without manual grading. When running in auto mode (the default when no --preset or --filter is specified), the tool performs a lightweight statistical analysis across sampled frames and builds a deterministic ffmpeg filter chain.

Sampling Frames with signalstats

The process begins in _sample_frame_stats (lines 99-110 of helpers/grade.py), which selects N frames (default 10) from the requested time range. The function constructs an ffmpeg command that pipes video through the signalstats filter followed by metadata=print:file=… to dump per-frame statistics into a temporary text file.

This approach avoids processing full-frame histograms, keeping the analysis fast and memory-efficient. The filter emits raw values at the source's native bit depth, which requires subsequent normalization.

Parsing Luma and Saturation Statistics

Once the temporary metadata file is generated, the parser (lines 131-152) extracts five critical values from signalstats:

• YMIN and YMAX – minimum and maximum luminance values
• YAVG – average luminance (brightness)
• SATAVG – average saturation
• YBITDEPTH – the bit depth of the decoded frames (e.g., 8 or 10)

These values populate four lists (y_avgs, y_mins, y_maxs, and sat_avgs) that represent the statistical distribution across the sampled frame range.

Normalizing Bit-Depth Values

Because signalstats reports values in the source's native bit depth (e.g., 0-255 for 8-bit video), the code reads YBITDEPTH (line 133) and normalizes all collected values to a 0-1 range by dividing by (2**bit_depth)-1 (lines 159-160).

This normalization yields three key metrics used for decision-making:

1. y_mean – the average luminance across samples
2. y_std – approximate luminance standard deviation derived from the sampled range (lines 171-172)
3. sat_mean – the average saturation level

Heuristic Decision Rules for Auto-Correction

The auto_grade_for_clip function (lines 176-250) applies algorithmic thresholds to determine necessary corrections:

Contrast adjustment – If the luminance range (y_std × 4) falls below 0.65, the system calculates a contrast boost of up to +8%.

Gamma correction – When y_mean is below 0.42, gamma is lifted up to +10% to brighten underexposed footage. If y_mean exceeds 0.60, gamma is slightly reduced to prevent clipping.

Saturation scaling – If sat_mean drops below 0.18, saturation increases by up to +4%. Conversely, if saturation exceeds 0.38, the value is modestly reduced.

All adjustments are clamped to ±8% (lines 246-249) to ensure the correction remains subtle and natural-looking.

Generating the ffmpeg Filter Chain

After determining the adjustments, the code assembles an ffmpeg eq= filter string (lines 252-264). If no adjustment exceeds a 0.5% change threshold, the filter string remains empty, and the video is copied without re-encoding to preserve quality.

The resulting filter is passed to apply_grade (lines 274-285), which executes ffmpeg with -vf <filter> to produce the final output.

CLI and Programmatic Usage

You can invoke the auto-grade analysis from the command line to preview the generated filter and statistics:

python helpers/grade.py --analyze sample.mp4

Typical output shows the raw metrics and calculated adjustments:

auto-grade stats:
  y_mean=0.37  y_range=0.48  sat_mean=0.21
  → contrast=1.070  gamma=1.080  sat=0.980
  → filter: eq=contrast=1.070:gamma=1.080:saturation=0.980

For integration into custom pipelines, import the grading functions directly:

from pathlib import Path
from helpers.grade import auto_grade_for_clip, apply_grade

# Analyze clip and generate filter

video_path = Path("assets/clip.mov")
filter_str, stats = auto_grade_for_clip(
    video_path, 
    start=5.0, 
    duration=12.3, 
    verbose=True
)

# Apply the correction

apply_grade(
    video_path, 
    Path("outputs/clip_autograded.mp4"), 
    filter_str
)

To access raw statistics without generating a filter, use the low-level sampling function:

from helpers.grade import _sample_frame_stats

stats = _sample_frame_stats(
    Path("clip.mp4"), 
    start=0, 
    duration=30, 
    n_samples=5
)
print(stats)  # {'y_mean': 0.41, 'y_std': 0.12, 'sat_mean': 0.23}

Summary

• The auto-grade feature resides in helpers/grade.py and uses ffmpeg's signalstats filter to extract per-frame luminance and saturation data.
• Ten frames are sampled by default to calculate y_mean, y_std, and sat_mean after normalizing for bit depth.
• Heuristic thresholds determine contrast, gamma, and saturation adjustments, clamped to ±8% to maintain subtle corrections.
• If adjustments are minimal (<0.5%), the video passes through without re-encoding; otherwise, an eq= filter is applied via apply_grade.

Frequently Asked Questions

What statistics does ffmpeg signalstats provide for auto-grading?

The signalstats filter provides YMIN, YMAX, YAVG (luma statistics), SATAVG (saturation), and YBITDEPTH (bit depth). According to the source code in helpers/grade.py, these values are extracted from a metadata file generated by ffmpeg and used to calculate normalized brightness and saturation metrics.

How does the auto-grade feature handle different video bit depths?

The code reads the YBITDEPTH value from signalstats and normalizes all luma and saturation values to a 0-1 range by dividing by (2**bit_depth)-1. This ensures consistent analysis whether the source is 8-bit, 10-bit, or higher.

When will auto-grade skip processing and copy the video without changes?

If the calculated adjustments for contrast, gamma, and saturation all fall below 0.5%, auto_grade_for_clip returns an empty filter string. In this case, the pipeline copies the video without re-encoding, preserving the original quality and saving processing time.

Can I adjust the number of frames sampled for analysis?

Yes. The _sample_frame_stats function accepts an n_samples parameter (default 10) that controls how many frames are analyzed across the specified time range. Fewer samples increase speed but may reduce statistical accuracy for clips with variable exposure.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}