How Timestamp Cues Are Handled and Prioritized During Frame Extraction in Claude‑Video

Timestamp cues supplied via --timestamps are parsed, validated, and pinned as mandatory frames before any other sampling logic runs, ensuring user‑requested moments always appear in the final output even when frame budgets are tight.

Claude‑Video's frame extraction pipeline gives explicit priority to user‑provided timestamps, treating them as cue frames that survive deduplication and budget constraints. This article breaks down the complete handling flow—from CLI argument parsing through final frame merging—based on the source code in bradautomates/claude-video.

Parsing and Validating User‑Provided Timestamps

The journey begins in skills/watch/scripts/frames.py with parse_timestamps at line 295, which transforms raw comma‑separated strings into a clean, de‑duplicated list of seconds.

cue_timestamps = parse_timestamps(args.timestamps)   # → [1.0, 3.0, …]

This function handles mixed formats including plain seconds (1, 3) and HH:MM:SS notation (00:01:15), normalizing everything to float seconds. The resulting list becomes the foundation for all subsequent cue‑frame logic.

Filtering to the Active Focus Window

Inside extract_at_timestamps, timestamps are clipped to the optional [start, end] range specified via --start and --end CLI flags. Anything outside this window is counted as dropped and reported in extraction metadata.

lo = start_seconds or 0.0
hi = end_seconds if end_seconds is not None else float("inf")
in_window = [t for t in requested if lo <= t <= hi]   # lines 51‑52

The dropped_out_of_window count in the returned metadata dict alerts users when their requested timestamps fall outside the current focus range.

Budget‑Aware Sampling of Cue Frames

Cue frames compete for the same max_frames budget as detail frames. When the number of in‑window timestamps exceeds available slots, Claude‑Video applies deterministic sampling via _even_indices:

if max_frames is not None and len(in_window) > max_frames:
    points = [in_window[i] for i in _even_indices(len(in_window), max_frames)]
else:
    points = in_window

This algorithm preserves the first and last cues while evenly distributing selections across the middle—maintaining temporal spread without arbitrary truncation.

Exact‑Frame Extraction with ffmpeg

Each retained timestamp triggers a precise ffmpeg capture at the exact second using -ss:

out.append({
    "index": len(out),
    "timestamp_seconds": t,
    "path": str(path),
    "reason": "transcript-cue",
})

Output files follow the naming convention cue_####.jpg, and every frame dict carries reason="transcript-cue" for traceability in downstream processing.

Merging Cue Frames with Detail Engine Output

The critical prioritization logic appears in skills/watch/scripts/frames.py at line 312 within merge_frames. Before the detail engine (keyframes, scene‑aware sampling, or uniform frames) runs, the available budget is reduced:

detail_budget = max_frames - len(cue_frames)

This reservation guarantees cue frames never compete for their own slots. After detail extraction completes, merge_frames performs a chronological union:

frames = merge_frames(frames, cue_frames)   # lines 27‑28 in watch.py

Cue frames are never dropped at this stage—they've already been accounted for in budget calculations.

CLI Reporting and Metadata

The final summary distinguishes cue frames from regular detail frames, surfacing any window‑related drops:

print(f"- **Cue frames:** {len(cue_frames)} at transcript-flagged timestamps "
      f"(transcript-cue{drop_note})")   # lines 5‑7 in watch.py

This transparency helps users verify their --timestamps inputs produced expected results.

Complete Usage Examples

Command‑Line Invocation

# Request frames at 1s and 3s alongside balanced detail sampling

claude-video watch "https://youtu.be/xyz" --detail balanced --timestamps 1,3

# Focus on a sub‑range while preserving cues at specific moments

claude-video watch "video.mp4" --start 00:00:30 --end 00:01:00 \
    --detail efficient --timestamps 31,45,58

Programmatic API Access

from skills.watch.scripts.frames import parse_timestamps, extract_at_timestamps
from pathlib import Path

# 1. Parse user string with mixed formats

cue_ts = parse_timestamps("1, 3, 00:01:15")

# 2. Extract cue frames with budget enforcement

cue_frames, meta = extract_at_timestamps(
    video_path="sample.mp4",
    out_dir=Path("/tmp/frames"),
    timestamps=cue_ts,
    max_frames=5,
)

print(cue_frames)   # [{'index': 0, 'timestamp_seconds': 1.0, ...}, ...]

print(meta)         # {'engine': 'timestamps', 'dropped_out_of_window': 0, ...}

Key Implementation Files

• skills/watch/scripts/frames.py — Core utilities: parse_timestamps, extract_at_timestamps, merge_frames, and _even_indices sampling helper.
• skills/watch/scripts/watch.py — CLI driver orchestrating cue extraction and frame merging.
• tests/test_timestamps.py — Unit tests validating parsing, window filtering, and cue‑frame behavior.
• tests/test_watch.py — End‑to‑end verification that cue frames appear in final reports.

Summary

• Parsing: parse_timestamps normalizes comma‑separated inputs to float seconds, supporting multiple time formats.
• Window filtering: Cues outside [start, end] ranges are tracked as dropped_out_of_window but don't fail extraction.
• Budget reservation: Cue frames are counted against max_frames before detail engine runs, guaranteeing their presence.
• Even sampling: _even_indices distributes cues deterministically when budgets force reduction.
• Chronological merge: merge_frames unions cue and detail frames without dropping previously retained cues.

Frequently Asked Questions

What happens if I provide more timestamps than my --max-frames budget allows?

Claude‑Video applies _even_indices sampling to spread selections evenly across your timestamp list, always keeping the first and last cues. This deterministic approach prioritizes temporal coverage over simple truncation.

Do timestamp cues work with the --start and --end focus window flags?

Yes, but cues outside the window are filtered and reported via dropped_out_of_window metadata. Only in‑window timestamps proceed to extraction and budget calculations.

Can I use HH:MM:SS format in --timestamps instead of raw seconds?

parse_timestamps accepts mixed formats including 00:01:15 notation, normalizing all inputs to float seconds internally.

Are cue frames visually different from regular detail frames in the output?

All frames are JPEG files, but cue frames use the cue_####.jpg naming pattern and carry reason="transcript-cue" in their metadata dicts for downstream identification.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}