# How Pinned Cue Frames Ensure Explicit Timestamp Requests Survive the Frame Cap

> Learn how pinned cue frames guarantee explicit timestamp requests survive the frame cap by being extracted first and reserved. Avoid dropped requests with this essential technique.

- Repository: [bradautomates/claude-video](https://github.com/bradautomates/claude-video)
- Tags: internals
- Published: 2026-08-10

---

**Cue frames are guaranteed to survive the frame cap because they are extracted first, reserved in the budget calculation, and merged without eviction—ensuring explicit timestamp requests are never dropped.**

In the `bradautomates/claude-video` repository, the **watch skill** handles two distinct frame types: **cue frames** (user-requested timestamps) and **detail frames** (automatically extracted scenes). The architecture prioritizes cue frames through a three-stage pipeline that reserves capacity before any automatic extraction occurs.

## The Three-Stage Protection Mechanism

### Stage 1: Extract Cue Frames Before Any Budget Enforcement

When `cue_timestamps` are provided, the system calls `extract_at_timestamps()` **immediately**—before the detail engine runs. This ensures explicit requests are satisfied first, independent of the `max_frames` limit.

In [`watch.py`](https://github.com/bradautomates/claude-video/blob/main/watch.py) (lines 76-80):

```python

# 1️⃣ Pull out any user-requested timestamps

if cue_timestamps and video_path:
    cue_frames, cue_meta = extract_at_timestamps(
        video_path,
        work / "frames",
        cue_timestamps,
        resolution=args.resolution,
        max_frames=max_frames,
        start_seconds=start_sec,
        end_seconds=end_sec,
    )
    # Log any timestamps that fell outside the focus range

    if cue_meta.get("dropped_out_of_window"):
        print("[watch] … cue timestamps outside the focus range — dropped", file=sys.stderr)

```

Only timestamps **outside the user-specified focus window** are dropped at this stage. The remaining cue frames are locked in and counted toward the final output.

### Stage 2: Reserve Budget Exclusively for Detail Extraction

The critical protection happens in the budget calculation. The system subtracts cue frame count from `max_frames`, creating a **separate detail budget** that cannot touch the reserved cue frames.

In [`watch.py`](https://github.com/bradautomates/claude-video/blob/main/watch.py) (lines 95-96):

```python

# 2️⃣ Reserve space for those frames before running the detail engine

detail_budget = max_frames if max_frames is None else max(0, max_frames - len(cue_frames))

```

This ensures:
- The detail engine operates on `detail_budget` only
- Cue frames remain **outside the scope of automatic extraction limits**
- Frame cap enforcement applies solely to scene-based frames

### Stage 3: Merge Without Eviction via `merge_frames()`

After detail extraction completes, the two lists are combined. The `merge_frames()` function in [`frames.py`](https://github.com/bradautomates/claude-video/blob/main/frames.py) treats cue frames as **pinned**—performing a simple union operation with no filtering or eviction logic.

In [`frames.py`](https://github.com/bradautomates/claude-video/blob/main/frames.py) (lines 312-319):

```python
def merge_frames(primary: list[dict], pinned: list[dict]) -> list[dict]:
    """
    Combine two frame lists into one chronological list and reindex 0..n-1.

    ``pinned`` frames (transcript cues) are never dropped — this is a plain
    union, so the cap is enforced upstream by reserving budget for the cues.
    """
    merged = sorted([*primary, *pinned], key=lambda f: f["timestamp_seconds"])
    for i, frame in enumerate(merged):
        frame["index"] = i
    return merged

```

The function sorts chronologically and reindexes—**never discarding elements**. The comment explicitly documents this contract: eviction prevention happens upstream through budget reservation, making the merge operation safe and simple.

## Complete Pipeline Example

Here's the full flow from [`watch.py`](https://github.com/bradautomates/claude-video/blob/main/watch.py) showing how explicit timestamp requests are protected end-to-end:

```python

# 1️⃣ Pull out any user‑requested timestamps

if cue_timestamps and video_path:
    cue_frames, cue_meta = extract_at_timestamps(
        video_path,
        work / "frames",
        cue_timestamps,
        resolution=args.resolution,
        max_frames=max_frames,
        start_seconds=start_sec,
        end_seconds=end_sec,
    )

# 2️⃣ Reserve space for those frames before running the detail engine

detail_budget = max_frames if max_frames is None else max(0, max_frames - len(cue_frames))

# …run the detail engine (keyframes, scene‑aware, etc.) using `detail_budget`…

# 3️⃣ Merge the two lists – pinned cue frames are never evicted

if cue_frames:
    frames = merge_frames(frames, cue_frames)

```

## Why This Design Guarantees Frame Survival

| Protection Layer | Mechanism | Code Location |
|------------------|-----------|---------------|
| **Temporal priority** | Cue extraction runs before detail engine | `watch.py:76-80` |
| **Budget isolation** | `detail_budget = max_frames - len(cue_frames)` | `watch.py:95-96` |
| **Non-destructive merge** | `merge_frames()` performs pure union | `frames.py:312-319` |

These layers ensure that **explicit timestamp requests are always honored**, even when the overall frame cap would otherwise force eviction. The frame cap applies only to automatically generated detail frames—the user's explicit selections bypass the constraint entirely through reservation.

## Key Source Files

| File | Purpose | Direct Link |
|------|---------|-------------|
| [`skills/watch/scripts/watch.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/watch.py) | Orchestrates cue extraction, budget reservation, and final merge | [View on GitHub](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/watch.py) |
| [`skills/watch/scripts/frames.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/frames.py) | Core frame utilities including `merge_frames()` | [View on GitHub](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/frames.py) |
| [`tests/test_timestamps.py`](https://github.com/bradautomates/claude-video/blob/main/tests/test_timestamps.py) | Unit tests verifying cue frame preservation under cap constraints | [View on GitHub](https://github.com/bradautomates/claude-video/blob/main/tests/test_timestamps.py) |

## Summary

- **Cue frames** (explicit timestamp requests) are extracted **first** and **reserved** in the frame budget
- The **detail budget** is calculated as `max(0, max_frames - len(cue_frames))`, isolating automatic extraction from pinned frames
- **`merge_frames()`** performs a non-destructive union, with the comment explicitly stating that eviction prevention happens upstream
- Cue frames dropped **only** if they fall outside the user-specified focus window—never due to frame cap pressure
- Unit tests in [`test_timestamps.py`](https://github.com/bradautomates/claude-video/blob/main/test_timestamps.py) verify this behavior under cap-constrained conditions

## Frequently Asked Questions

### What happens if cue timestamps exceed the max_frames value entirely?

The `detail_budget` calculation uses `max(0, max_frames - len(cue_frames))`, which floors at zero. This means the detail engine receives no budget and produces no automatic frames, but all valid cue frames are still extracted and preserved. The `merge_frames()` function returns only the cue list in this case.

### Can cue frames be dropped for any reason other than the frame cap?

Yes—cue frames are filtered solely by `extract_at_timestamps()` if their timestamps fall outside the `start_seconds`/`end_seconds` focus window. This is logged via `cue_meta.get("dropped_out_of_window")`. These drops are **user-controlled range constraints**, not automatic eviction.

### Why is the frame cap passed to `extract_at_timestamps()` if cue frames are supposed to bypass it?

The `max_frames` parameter in `extract_at_timestamps()` is used for input validation and logging purposes, not for limiting cue frame output. The actual enforcement happens downstream through the `detail_budget` reservation and the non-destructive `merge_frames()` operation.

### How does `merge_frames()` handle timestamp collisions between cue and detail frames?

The function sorts by `timestamp_seconds` and reindexes sequentially. If two frames share identical timestamps, Python's stable sort preserves original order. No deduplication occurs—both frames appear in the final output, maintaining the contract that pinned frames are never dropped.