# How the Frame Budget Algorithm Scales with Video Duration in Balanced Mode

> Discover how the frame budget algorithm scales with video duration in balanced mode. Learn how it caps frames and adjusts sampling rates to manage processing overhead efficiently.

- Repository: [bradautomates/claude-video](https://github.com/bradautomates/claude-video)
- Tags: performance
- Published: 2026-08-08

---

**In balanced mode, the frame budget algorithm caps extraction at 100 frames total and dynamically lowers the sampling rate as video duration increases to maintain consistent processing overhead.**

The `bradautomates/claude-video` repository implements an intelligent **balanced mode** that manages computational resources when analyzing video content. This mode employs a frame budget algorithm that scales inversely with video length, ensuring short clips retain temporal detail while enforcing strict limits on long-form content.

## Frame Cap Configuration

The balanced mode enforces a hard limit of **100 frames** per video, defined in [`skills/watch/scripts/config.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/config.py). The `frame_cap()` function returns this ceiling when called with the `"balanced"` detail parameter.

```python

# skills/watch/scripts/config.py (lines 65-74)

def frame_cap(detail_mode: str) -> int:
    caps = {
        "low": 50,
        "balanced": 100,  # Maximum frame budget

        "high": 200
    }
    return caps.get(detail_mode, 100)

```

This constant frame budget ensures predictable memory usage and API processing costs regardless of input video length.

## Duration-Based Scaling Rules

The scaling logic resides in [`skills/watch/scripts/frames.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/frames.py) within the `auto_fps()` function (lines 22-38). This function calculates a target frame count based on video duration, then derives the FPS required to hit that target without exceeding **MAX_FPS = 2.0**.

The algorithm applies five distinct duration tiers:

- **≤ 30 seconds**: Targets `min(100, max(12, ⌈duration⌉))` frames (12–30 frames), yielding approximately **0.4–1 fps** (roughly one frame per second)
- **30–60 seconds**: Fixed target of **40 frames**, resulting in **0.67–1.33 fps**
- **60–180 seconds**: Fixed target of **60 frames**, resulting in **0.33–1 fps**
- **180–600 seconds**: Fixed target of **80 frames**, resulting in **0.13–0.44 fps**
- **> 600 seconds (10 minutes)**: Uses the full **100-frame budget**, calculating FPS as `100 / duration` (very low sampling rates for long videos)

As video duration increases, the algorithm maintains the frame budget by proportionally decreasing the extraction frequency.

## Implementation Workflow

The orchestration logic in [`skills/watch/scripts/watch.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/watch.py) coordinates the frame budget application. When processing begins, the script:

1. Retrieves the configured cap via `frame_cap("balanced")` → `100`
2. Passes the video duration and cap to `auto_fps(duration, max_frames)`
3. Receives the calculated FPS and target frame count
4. Falls back to scene detection if the detected scenes yield fewer frames than the budget; otherwise applies uniform FPS extraction

This workflow ensures that the **100-frame ceiling** is respected while preferring scene-based segmentation when temporally sparse.

## Practical Usage

Calculate the exact FPS budget programmatically using the internal utilities:

```python
from skills.watch.scripts.config import frame_cap
from skills.watch.scripts.frames import auto_fps

detail = "balanced"
max_frames = frame_cap(detail)          # Returns 100

fps, target = auto_fps(45, max_frames)  # 45-second video

print(f"FPS: {fps:.2f}, Target frames: {target}")

# Output: FPS: 0.89, Target frames: 40

```

From the command line, balanced mode is the default detail setting:

```bash
python -m skills.watch.scripts.watch my_video.mp4 --detail balanced

```

The CLI automatically invokes `auto_fps()` with the 100-frame cap, applying the duration-based scaling rules without requiring manual FPS calculation.

## Summary

- **Hard cap**: Balanced mode enforces a strict 100-frame limit via `frame_cap()` in [`skills/watch/scripts/config.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/config.py)
- **Inverse scaling**: The `auto_fps()` function decreases sampling rates as duration increases to maintain the budget
- **Five tiers**: Duration thresholds at 30s, 60s, 180s, and 600s trigger specific frame targets (12-30, 40, 60, 80, 100)
- **FPS ceiling**: Extraction never exceeds 2.0 FPS, even for very short clips
- **Scene preference**: If scene detection produces fewer frames than the budget, the engine uses scene boundaries instead of uniform sampling

## Frequently Asked Questions

### What is the maximum number of frames extracted in balanced mode?

The algorithm caps extraction at **100 frames** regardless of video length. This limit is defined in [`skills/watch/scripts/config.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/config.py) and applies uniformly to all videos processed in balanced mode.

### How does the algorithm handle videos longer than 10 minutes?

For videos exceeding 600 seconds, the algorithm utilizes the full 100-frame budget. It calculates FPS as `100 / duration`, resulting in very sparse sampling (e.g., a 20-minute video extracts at approximately 0.083 fps, or one frame every 12 seconds).

### What happens if scene detection finds fewer frames than the budget?

When the scene detection engine identifies fewer keyframes than the calculated budget, the system prefers the scene-based frames over uniform FPS extraction. This ensures that natural transition points are preserved even if the total count falls below the 100-frame target.

### Why is the FPS capped at 2.0 in balanced mode?

The **MAX_FPS = 2.0** constant in [`skills/watch/scripts/frames.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/frames.py) prevents excessive sampling of short clips, ensuring that the frame budget is distributed intelligently rather than extracting redundant near-duplicate frames from high-frame-rate source material.