# How the Transcript-Cue System Pins Specific Timestamps Against the Frame Budget Cap

> Discover how the transcript-cue system preserves user timestamps within frame budget caps. Learn how cue frames are extracted, counted, and protected from eviction.

- Repository: [bradautomates/claude-video](https://github.com/bradautomates/claude-video)
- Tags: deep-dive
- Published: 2026-08-08

---

**The transcript-cue system guarantees user-requested timestamps survive frame limits by extracting cue frames first, subtracting their count from the global `max_frames` allowance, and preventing the detail engine from evicting them.**

The `bradautomates/claude-video` repository implements a sophisticated transcript-cue feature that lets users request exact frames at specific timestamps derived from captions or Whisper transcripts. This system ensures these critical moments are preserved even when operating under strict frame budget constraints. By pinning cue frames before running any detail extraction algorithms, the codebase maintains semantic fidelity to user-specified moments while respecting computational limits.

## How the Transcript-Cue System Reserves Frame Budget

The pinning mechanism operates through a strict priority pipeline where transcript cues are processed before any automated frame selection occurs. This sequence ensures that user intent takes precedence over algorithmic sampling.

### Parsing User-Provided Timestamps

The workflow begins in [`skills/watch/scripts/watch.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/watch.py), where the `parse_timestamps` function converts user input into a list of seconds. Located at [line 81](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/watch.py#L81), this utility handles the `--timestamps` argument and validates the input format before passing the normalized values downstream.

### Extracting Cue Frames Before Detail Analysis

Once timestamps are parsed, the system immediately extracts frames at those specific moments via `extract_at_timestamps` (lines 76-79). This function:
- Discards timestamps outside the focus window
- Respects the user-supplied `max_frames` cap through even sampling when necessary
- Returns `cue_frames` that are stored separately from detail-engine output

Because this extraction happens **before** any keyframe or scene-detection algorithms run, the cue frames are guaranteed to exist regardless of what the detail engine subsequently selects.

## Budget Allocation and Pinning Mechanism

After cue extraction, the system performs a critical budget reservation step that mathematically prevents frame over-allocation.

### Calculating the Detail Engine Budget

At lines 95-96 in [`watch.py`](https://github.com/bradautomates/claude-video/blob/main/watch.py), the remaining budget for the detail engine is calculated as:

```python
detail_budget = max_frames - len(cue_frames)

```

This subtraction ensures that the **pinned frames consume part of the total allowance**, leaving only the residual capacity for automated frame selection. When the detail engine (whether using efficient keyframes or balanced scene-aware extraction) receives its `max_frames` argument at lines 104-111, it operates within this constrained `detail_budget` and cannot steal slots from the cue frames.

### Preventing Cue Frame Eviction

The final protection occurs in [`skills/watch/scripts/frames.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/frames.py) within the `merge_frames` function (lines 12-22). This utility unions the detail-engine frames with the cue frames without filtering:

```python
def merge_frames(primary: list[dict], pinned: list[dict]) -> list[dict]:
    """Combine two frame lists … ``pinned`` frames (transcript cues) are never dropped."""
    merged = sorted([*primary, *pinned], key=lambda f: f["timestamp_seconds"])
    for i, frame in enumerate(merged):
        frame["index"] = i
    return merged

```

Because the function simply concatenates and sorts the two lists, cue frames persist in the final output even if the detail engine produced more frames than expected.

## Handling Budget Overages with Even Sampling

When users request more timestamps than the `max_frames` cap allows, the system applies intelligent down-sampling rather than failing. In [`frames.py`](https://github.com/bradautomates/claude-video/blob/main/frames.py) (lines 54-57), the `extract_at_timestamps` function implements even sampling:

```python
if max_frames is not None and len(in_window) > max_frames:
    points = [in_window[i] for i in _even_indices(len(in_window), max_frames)]

```

This algorithm preserves the first and last requested timestamps while distributing selections evenly across the remaining cues. The metadata returned by `extract_at_timestamps` also records any timestamps that fell outside the focused range in `dropped_out_of_window`, ensuring transparency when constraints force omissions.

## Code Implementation Example

The following demonstrates the complete workflow from command-line invocation through budget reservation:

```bash

# Request two specific timestamps while limiting total frames to 8

watch https://youtu.be/abc123 --timestamps "5.0,12.3" --max-frames 8

```

Inside [`watch.py`](https://github.com/bradautomates/claude-video/blob/main/watch.py), the orchestration logic executes:

```python

# Parse and extract cue frames first

cue_timestamps = parse_timestamps(args.timestamps)               # → [5.0, 12.3]

cue_frames, cue_meta = extract_at_timestamps(
    video_path, work / "frames", cue_timestamps,
    max_frames=args.max_frames
)                                                               # → 2 cue frames

# Reserve remaining budget for detail engine

detail_budget = max_frames - len(cue_frames)                    # → 6 remaining frames

frames, frame_meta = extract_keyframes(..., max_frames=detail_budget)

# Merge without dropping pinned frames

frames = merge_frames(frames, cue_frames)                       # final list = 8 frames

```

## Summary

- **Early extraction** guarantees cue frames exist before detail algorithms run, preventing eviction by scene-detection logic.
- **Budget reservation** via `detail_budget = max_frames - len(cue_frames)` mathematically isolates cue frame capacity from detail-engine allocation.
- **Even sampling** automatically handles cases where user requests exceed the cap, preserving temporal distribution through first/last spacing.
- **Immutable merging** in `merge_frames` ensures pinned timestamps survive the final frame compilation regardless of detail-engine output volume.
- **Metadata transparency** reports out-of-window drops without counting them against the frame budget.

## Frequently Asked Questions

### What happens if I request more timestamps than the max_frames limit?

The system applies even sampling to distribute selections across your requested range while preserving the first and last timestamps. As implemented in [`frames.py`](https://github.com/bradautomates/claude-video/blob/main/frames.py), the `_even_indices` helper selects evenly spaced indices from the input list, ensuring representative coverage without exceeding the cap.

### Can the detail engine accidentally remove my pinned cue frames?

No. The detail engine operates on a separate `detail_budget` calculated **after** cue extraction. Because `merge_frames` simply unions the two lists without filtering, and cue frames are extracted first, the detail engine has no mechanism to evict or overwrite pinned timestamps.

### How does the system handle timestamps outside the video focus window?

Timestamps falling outside the specified focus window are recorded in the `dropped_out_of_window` metadata field but are **not** counted against the frame budget. Only timestamps within the window compete for the `max_frames` allocation.

### Where is the frame budget logic implemented in the source code?

The primary budget calculation occurs in [`skills/watch/scripts/watch.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/watch.py) at lines 95-96, where `detail_budget` is derived by subtracting the cue frame count from `max_frames`. The enforcement happens at lines 104-111, where this budget is passed as the `max_frames` argument to the chosen detail extraction engine.