# What Mathematical Analysis Does Auto Color Grading Use? A Deep Dive into video-use's FFmpeg Pipeline

> Discover the mathematical analysis behind auto color grading. video-use employs a three-step pipeline: frame sampling, luma/saturation normalization, and filter correction for stunning visuals.

- Repository: [Browser Use/video-use](https://github.com/browser-use/video-use)
- Tags: deep-dive
- Published: 2026-08-06

---

**The auto-color-grading feature in browser-use/video-use uses a three-step statistical pipeline: frame sampling at adaptive FPS, luma/saturation normalization, and decision-rule-based filter correction.**

Video color grading often relies on subjective manual adjustment. The `video-use` project implements an **auto-grade** mode that automates this process through rigorous mathematical analysis of per-frame video statistics. This article examines the exact calculations performed in [`helpers/grade.py`](https://github.com/browser-use/video-use/blob/main/helpers/grade.py), tracing how raw performance numbers become precise `ffmpeg` correction filters.

## How Auto Color Grading Samples Video Frames

The grading pipeline begins with `_sample_frame_stats`, which controls frame extraction density through an adaptive FPS calculation.

Rather than analyzing every frame—which would be computationally wasteful for long clips—the algorithm targets a fixed sample count. The FPS formula ensures consistent statistical power regardless of input duration:

```python
fps = max(0.5, min(n_samples / max(duration, 0.1), 10.0))

```

This clamps the sampling rate between **0.5 and 10 FPS**, defaulting to `n_samples=10` frames total. For a 20-second clip, this yields `10/20 = 0.5 FPS`; for a 1-second clip, it hits the 10 FPS ceiling to gather enough data.

The `ffmpeg signalstats` filter extracts per-frame metadata without full decode, making this step efficient enough for real-world automation pipelines.

## Statistical Normalization: From Raw Values to Normalized Metrics

Once frames are sampled, the parser extracts four key metrics from `signalstats` output: `YMIN`, `YMAX`, `YAVG` (luma statistics), and `SATAVG` (saturation). These raw values must be normalized to the video's bit depth for cross-format compatibility.

The normalization divisor calculates as:

```python
max_val = 2 ** bit_depth - 1

```

For standard 8-bit video, `max_val = 255`; for 10-bit, `max_val = 1023`. The pipeline then computes three **normalized statistics** used in downstream decisions:

| Statistic | Calculation | Purpose |
|-----------|-------------|---------|
| **Mean luma** | `y_mean = avg(YAVG) / max_val` | Overall brightness level |
| **Luma range** | `y_range = (avg(YMAX) - avg(YMIN)) / max_val` | Dynamic range utilization |
| **Mean saturation** | `sat_mean = avg(SATAVG) / max_val` | Color intensity |

An approximate **standard deviation** derives from range under normality assumptions: `y_std = y_range / 4`. This estimation avoids costly per-pixel variance calculation while providing sufficient spread information for contrast decisions.

## Decision Rules: Converting Statistics to Filter Parameters

The final transformation applies domain-specific thresholds to generate `ffmpeg eq` filter adjustments. These rules encode cinematographic best practices into executable logic.

### Contrast Adjustment

The target luma range is approximately **0.72** (normalized). When `y_range` falls below this threshold, the algorithm calculates a contrast boost:

```python
contrast = 1.0 + (0.72 - y_range) * adjustment_factor

```

Values clamp to reasonable bounds (typically 0.5–2.0) to prevent extreme artifacts.

### Gamma Correction

Mean luma deviation from neutral gray (0.5 normalized) triggers gamma adjustment using logarithmic mapping. Dark clips (`y_mean < 0.5`) receive negative gamma; bright clips receive positive values.

### Saturation Scaling

`sat_mean` comparison against a reference level (often 0.5 or context-dependent) produces multiplicative saturation adjustment. Under-saturated footage gets boosted; over-saturated material gets pulled back.

All three parameters feed into a single `eq=contrast=X:gamma=Y:saturation=Z` filter string for efficient `ffmpeg` application.

## Why This Mathematical Pipeline Works

The auto-grading approach succeeds because it decouples **measurement** from **correction**. By analyzing actual frame statistics rather than histograms or presets, it adapts to content rather than assuming uniform distributions.

The use of `signalstats` metadata extraction—rather than pixel buffer analysis—maintains performance without sacrificing accuracy. The normal distribution assumption for standard deviation estimation holds sufficiently well for natural imagery, where luma values cluster centrally.

## Summary

- **Adaptive sampling**: FPS calculation in `_sample_frame_stats` (lines 96–100) ensures 10-frame analysis regardless of clip duration
- **Bit-depth normalization**: All metrics divide by `2^depth - 1` for format-agnostic comparison
- **Three core statistics**: Mean luma, luma range, and mean saturation drive all decisions
- **Approximately normal**: Standard deviation estimated as `y_range / 4` avoids expensive variance computation
- **Cinematographic rules**: Contrast targets 0.72 range; gamma and saturation respond to mean deviations

## Frequently Asked Questions

### What video bit depths does auto color grading support?

The normalization logic in [`helpers/grade.py`](https://github.com/browser-use/video-use/blob/main/helpers/grade.py) dynamically detects bit depth from `signalstats` metadata, supporting any depth where `max_val = 2^n - 1` applies. Common formats include 8-bit (max_val=255), 10-bit (1023), and 12-bit (4095). All statistics scale proportionally.

### Why use `y_range / 4` instead of calculating true standard deviation?

True standard deviation requires second-moment calculation across all pixels. The range-to-SD approximation assumes roughly normal luma distribution, which holds for most natural footage. This trade-off reduces computational cost by 90%+ while maintaining sufficient precision for contrast decisions.

### Can the default 10-frame sample count be adjusted?

The `n_samples` parameter in `_sample_frame_stats` is configurable at call time. Higher values improve statistical stability for clips with extreme frame-to-frame variation; lower values accelerate processing for batch workflows. The 0.5–10 FPS clamping prevents degenerate cases regardless of sample count.

### How does auto color grading handle already-correct footage?

The decision rules include dead zones where no adjustment occurs. When `y_range` already exceeds 0.72, `y_mean` sits near 0.5, and `sat_mean` matches reference, all correction factors evaluate to 1.0 (neutral). The resulting `eq` filter applies identity transformation, leaving well-graded content untouched.