# How the bradautomates/claude-video Frame Budget System Works: Cost-Controlled Video Processing

> Uncover how the bradautomates/claude-video frame budget system optimizes frame extraction for controlled LLM token costs. Learn about automatic rate calculation and perceptual deduplication.

- Repository: [bradautomates/claude-video](https://github.com/bradautomates/claude-video)
- Tags: internals
- Published: 2026-07-07

---

**The bradautomates/claude-video frame budget system automatically calculates optimal frame extraction rates based on video duration, caps extraction at configurable limits, and applies perceptual deduplication to deliver predictable token costs for downstream LLM processing.**

The bradautomates/claude-video repository implements an intelligent budgeting mechanism that prevents runaway token costs when processing video content through Claude or other LLMs. Located primarily in [`skills/watch/scripts/frames.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/frames.py), this system dynamically adjusts frames-per-second (FPS) sampling and enforces hard caps to ensure that both short clips and long videos remain within predictable processing budgets.

## Understanding the Three-Stage Budget Architecture

The frame budget system operates through a pipeline of three distinct stages: target calculation, constrained extraction, and intelligent deduplication. Each stage ensures that the final frame count aligns with the video's duration while respecting the global `max_frames` parameter (default: 100).

### Stage 1: Calculating Target Frame Counts

The system employs two specialized functions to determine appropriate sampling rates: `auto_fps()` for full-video analysis and `auto_fps_focus()` for restricted time ranges.

#### Full-Video Budget Calculation with auto_fps()

For complete video scans, `auto_fps()` in [`skills/watch/scripts/frames.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/frames.py) implements a tiered budgeting strategy that allocates frames based on duration:

- **Short clips (< 30 seconds)**: Generous budget of up to 12 frames plus one frame per second
- **30-60 seconds**: Capped at 40 frames
- **1-3 minutes**: Capped at 60 frames  
- **3-10 minutes**: Capped at 80 frames
- **Longer videos**: Limited to the user-specified `max_frames` value

The function also enforces `MAX_FPS = 2.0`, ensuring extraction never exceeds two frames per second regardless of duration.

```python

# skills/watch/scripts/frames.py – auto_fps() implementation

def auto_fps(duration_seconds: float, max_frames: int = 100) -> tuple[float, int]:
    if duration_seconds <= 30:
        target = min(max_frames, max(12, int(round(duration_seconds)))
    elif duration_seconds <= 60:
        target = min(max_frames, 40)
    # ... additional duration tiers

    return _clamp_fps(target / duration_seconds, duration_seconds, max_frames)

```

#### Focused Segment Budget Calculation with auto_fps_focus()

When users specify time ranges using `--start` and `--end` parameters, `auto_fps_focus()` applies a steeper sampling rate to preserve detail in the restricted window. For segments under 5 seconds, it allocates up to 10 frames or 6 frames per second, whichever is higher.

```python

# skills/watch/scripts/frames.py – auto_fps_focus() implementation

def auto_fps_focus(duration_seconds: float, max_frames: int = 100) -> tuple[float, int]:
    if duration_seconds <= 5:
        target = min(max_frames, max(10, int(round(duration_seconds * 6))))
    # ... additional logic for longer focused segments

    return _clamp_fps(target / duration_seconds, duration_seconds, max_frames)

```

### Stage 2: Constrained Frame Extraction with ffmpeg

Once the target FPS is calculated, the `extract()` function passes both the sampling rate and global `max_frames` limit to an `ffmpeg` subprocess. The `-frames:v` parameter acts as a hard ceiling, automatically truncating extraction if the FPS calculation yields more frames than the budget allows.

```python

# skills/watch/scripts/frames.py – extract() method

cmd = [
    "ffmpeg", "-hide_banner", "-loglevel", "error", "-y",
    "-i", str(Path(video_path).resolve()),
    "-vf", f"fps={fps},{_scale_filter(resolution)}",
    "-frames:v", str(max_frames), "-q:v", "4", output_pattern,
]

```

This dual-layer enforcement ensures that even if duration calculations drift, the physical extraction halts at the defined budget boundary.

### Stage 3: Perceptual Deduplication and Final Sampling

After extraction, frames undergo `dedupe_perceptual()` to eliminate visually redundant content before final budget enforcement. This process:

1. Generates 16x16 grayscale thumbnails (`DEDUP_THUMB = 16`) for each frame
2. Computes mean-pixel deltas between consecutive frames
3. Drops frames falling below `DEDUP_THRESHOLD = 2.0`

The surviving frames then pass through `_even_sample()`, which performs stratified sampling to reach the final target count while preserving the first and last frames for temporal context.

```python

# skills/watch/scripts/frames.py – deduplication and sampling logic

def dedupe_perceptual(candidates, threshold=DEDUP_THRESHOLD):
    # ... thumbnail generation and delta calculation

    return _dedupe_by_deltas(candidates, thumbs, threshold)

def _even_sample(candidates, n):
    selected = [candidates[i] for i in _even_indices(len(candidates), n)]
    # ... return evenly distributed frame set

```

## Practical Implementation Examples

### Running a Full-Video Extraction

Execute the default budget calculation against an entire video file:

```bash
python -m skills.watch.scripts.frames /path/to/video.mp4 /tmp/frames

```

This automatically invokes `auto_fps()` to determine the appropriate sampling rate and extracts frames according to the duration-based budget tiers.

### Extracting Focused Segments

For analyzing specific time ranges with higher sampling density:

```bash
python -m skills.watch.scripts.frames \
    /path/to/video.mp4 /tmp/frames --start 1:30 --end 2:00

```

This triggers `auto_fps_focus()`, applying the steeper budget curve suitable for short temporal windows while maintaining the same `max_frames` safety cap.

## Summary

- **Duration-based budgeting**: The system automatically allocates more frames per second to short videos while capping long videos at fixed ceilings (40-80 frames depending on length).
- **Dual enforcement**: Budget limits are enforced both mathematically in `auto_fps()` and physically via `ffmpeg -frames:v` parameters.
- **Perceptual deduplication**: Near-identical frames are eliminated using 16x16 grayscale thumbnails before final sampling, ensuring budget slots hold unique visual information.
- **Configurable constants**: Key thresholds like `MAX_FPS = 2.0` and `DEDUP_THRESHOLD = 2.0` reside in [`skills/watch/scripts/config.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/config.py) for easy adjustment.
- **Range-specific optimization**: The `auto_fps_focus()` function provides denser sampling when analyzing video segments specified by `--start` and `--end` timestamps.

## Frequently Asked Questions

### What is the maximum FPS limit in the frame budget system?

The bradautomates/claude-video frame budget system enforces a hard ceiling of `MAX_FPS = 2.0` frames per second, as defined in [`skills/watch/scripts/config.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/config.py). This limit applies regardless of video duration or target calculations, ensuring that even very short clips never exceed two extracted frames per second.

### How does the system handle short videos versus long videos?

Short videos receive preferential treatment with denser sampling: clips under 30 seconds receive up to 12 frames plus one frame per second, while segments under 5 seconds (when using focused mode) receive up to 6 frames per second. Long videos transition to fixed caps: 40 frames for 30-60 seconds, 60 frames for 1-3 minutes, and 80 frames for up to 10 minutes, preventing excessive token costs from extended content.

### What is perceptual deduplication and how does it affect the frame budget?

Perceptual deduplication is a post-extraction optimization in `dedupe_perceptual()` that removes visually redundant frames before final budget enforcement. It creates 16x16 grayscale thumbnails of each extracted frame and compares mean-pixel deltas; frames with differences below `DEDUP_THRESHOLD = 2.0` are discarded. This ensures the final frame budget contains only visually distinct content, maximizing information density per allocated token.

### How do I extract frames from a specific time range using the budget system?

Specify `--start` and `--end` timestamps in `HH:MM:SS` or `MM:SS` format when invoking the frames module. This triggers `auto_fps_focus()` instead of `auto_fps()`, applying a steeper sampling curve appropriate for the restricted duration while maintaining the same `max_frames` safety cap. For example: `python -m skills.watch.scripts.frames video.mp4 /output --start 1:30 --end 2:00`.