How the Auto-Grade Algorithm Analyzes Brightness, Contrast, and Saturation per Clip

The auto-grade algorithm in browser-use/video-use samples frames using FFmpeg's signalstats filter, extracts per-frame luma and saturation metrics, then applies bounded linear transformations to generate an FFmpeg eq filter that corrects underexposed, flat, or desaturated clips while keeping adjustments within ±8%.

The browser-use/video-use repository provides a data-driven approach to color correction through its auto-grade feature. Located in helpers/grade.py, the algorithm performs per-clip analysis to automatically adjust brightness, contrast, and saturation without manual intervention. This article breaks down exactly how the code extracts statistical metrics from video frames and translates them into precise FFmpeg filter parameters.

Frame Sampling and Statistical Extraction

The analysis pipeline begins with _sample_frame_stats in helpers/grade.py, which extracts a representative set of frames from the requested time range.

FFmpeg signalstats Integration

The algorithm uses FFmpeg's signalstats filter to collect per-frame metadata. By default, it samples 10 evenly-spaced frames across the clip duration, calculating the sampling rate with fps = max(0.5, min(n_samples / duration, 10.0)) to prevent over-sampling (lines 97-100). This filter outputs raw integers for:

• YAVG – Average luma (brightness)
• YMIN/YMAX – Minimum and maximum luma values
• SATAVG – Average saturation
• YBITDEPTH – Native bit depth for normalization

Normalization and Metric Calculation

The code parses the temporary metadata file and converts raw values to a 0-1 floating-point range based on the native bit-depth (lines 84-94). From these normalized values, the algorithm computes three key metrics:

• y_mean – Average luma representing overall brightness
• y_std – Estimated luma standard deviation derived from the observed range (y_range / 4), serving as a contrast proxy
• sat_mean – Average saturation across the sampled frames

These statistics feed directly into the decision logic within auto_grade_for_clip.

Decision Rules for Brightness, Contrast, and Saturation

The auto_grade_for_clip function (lines 176-250) applies specific thresholds to determine when correction is necessary. Each parameter receives a subtle adjustment mapped linearly across defined ranges.

Contrast Adjustment Logic

Low contrast is detected when the luma range (y_range) falls below 0.65, indicating a flat image. The algorithm calculates a contrast boost by linearly mapping the range [0.50, 0.65] → [1.08, 1.03] (lines 176-222). Clips with adequate range receive a minimal baseline adjustment of 1.03. All contrast values are clamped to [0.94, 1.08].

Gamma (Brightness) Correction

For underexposed clips where y_mean is below 0.42, the algorithm applies a gamma lift by mapping [0.30, 0.42] → [1.10, 1.02] (lines 224-231). Over-exposed content exceeding 0.60 receives a small pull-back to 0.97. Gamma adjustments are bounded to [0.94, 1.10].

Saturation Boost and Reduction

Saturation analysis targets the sat_mean value. If saturation falls below 0.18, a modest boost of 1.04 is applied; if it exceeds 0.38, a reduction to 0.96 occurs (lines 235-244). The default neutral adjustment is 0.98, accounting for the tendency of consumer videos toward slight oversaturation. Saturation limits are [0.94, 1.06].

Building the FFmpeg Filter String

After calculating adjustments, the algorithm constructs an FFmpeg eq filter string in build_filter (lines 52-63). Only non-neutral values are included:

# Example generated filter string

eq=contrast=1.045:gamma=1.067:saturation=0.982

If all metrics fall within acceptable ranges, the function returns an empty string, and render.py copies the clip without re-encoding (lines 220-250). This ensures the "clean, not graded" aesthetic while avoiding unnecessary processing overhead.

Implementing Per-Clip Color Correction

You can analyze and apply auto-grade adjustments programmatically or via command line.

Python API Usage

from pathlib import Path
from helpers.grade import auto_grade_for_clip, apply_grade

# Analyze a 10-second segment starting at 30 seconds

clip_path = Path("src.mp4")
filter_str, stats = auto_grade_for_clip(
    clip_path, 
    start=30.0, 
    duration=10.0, 
    verbose=True
)

# Output: Statistical breakdown and filter string like "eq=contrast=1.03:gamma=1.05"

# Apply the correction (copies file if filter_str is empty)

output_path = Path("src_graded.mp4")
apply_grade(clip_path, output_path, filter_str)

Command Line Interface

# Auto-grade mode (default)

python helpers/grade.py input.mp4 -o output.mp4

# Analysis only (no rendering)

python helpers/grade.py --analyze input.mp4

# Use manual preset instead of auto

python helpers/grade.py --preset warm_cinematic -o out.mp4 input.mp4

Summary

• The auto-grade algorithm resides in helpers/grade.py and processes clips through three stages: frame sampling, statistical extraction, and decision-based filter construction.
• Frame sampling uses FFmpeg's signalstats with a default of 10 samples and calculated FPS to avoid over-sampling.
• Brightness translates to gamma adjustments ranging from 0.94 to 1.10 based on mean luma values.
• Contrast calculations rely on luma range analysis, with boosts up to 1.08 for flat images.
• Saturation receives corrections between 0.94 and 1.06 depending on average saturation levels.
• All adjustments respect an ±8% clamp to maintain a natural, ungraded appearance.

Frequently Asked Questions

What FFmpeg filter does the auto-grade algorithm use for statistical analysis?

The algorithm uses FFmpeg's signalstats filter to extract per-frame metadata including YAVG (average luma), YMIN/YMAX (luma range), and SATAVG (average saturation). This filter writes data to a temporary file that the Python code parses to calculate normalized brightness, contrast, and saturation metrics.

How does the algorithm handle clips that are already well-exposed?

When a clip falls within acceptable ranges—meaning y_mean is between 0.42 and 0.60, y_range exceeds 0.65, and sat_mean is between 0.18 and 0.38—the algorithm returns an empty filter string. According to the implementation in auto_grade_for_clip, this triggers a direct copy operation in helpers/render.py without re-encoding, preserving the original color profile.

What are the maximum adjustment limits for brightness, contrast, and saturation?

The auto-grade enforces strict bounds to prevent extreme corrections: contrast is clamped to [0.94, 1.08], gamma (brightness) to [0.94, 1.10], and saturation to [0.94, 1.06]. These limits are hardcoded in lines 246-250 of helpers/grade.py to ensure a "clean, not graded" aesthetic.

Can I apply auto-grade to specific time ranges within a video?

Yes. The auto_grade_for_clip function accepts start and duration parameters (in seconds) to analyze specific segments. When integrated into the rendering pipeline via helpers/render.py, this allows per-clip grading within an EDL (Edit Decision List) where individual segments can be flagged with the "auto" grade attribute.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}