# How Claude Video Achieves Chronological Order with Timestamps in Frame Extraction

> Discover how Claude Video ensures chronological frame order. Learn about precise timestamp computation, physical sequence sorting, and timestamp-based merging for accurate video analysis.

- Repository: [bradautomates/claude-video](https://github.com/bradautomates/claude-video)
- Tags: internals
- Published: 2026-07-13

---

**Claude Video extracts frames using ffmpeg and preserves chronological order by computing precise timestamps for each frame, sorting files by physical sequence, and re-indexing mixed sources using timestamp-based merging.**

Claude Video is an open-source video processing framework that maintains temporal accuracy across frame extraction pipelines. The system ensures chronological integrity by assigning precise timestamps during capture, enforcing physical file ordering, and implementing robust merging algorithms. This timestamp-centric approach guarantees that downstream transcription and analysis components receive frames in true chronological sequence regardless of extraction method.

## Timestamp Capture During Frame Extraction

Claude Video employs different timestamp strategies depending on the extraction mode, but both methods yield temporally accurate frame metadata that preserves the original video sequence.

### Calculating Timestamps for Uniform Extraction

For uniform frame extraction via the `extract()` function, timestamps are computed mathematically from the start offset and frame index. In [`skills/watch/scripts/frames.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/frames.py), the system calculates each frame's temporal position using:

```python
"timestamp_seconds": round(offset + (i / fps if fps > 0 else 0.0), 2)

```

This computation ensures that every extracted frame receives a precise timestamp relative to the video start, with sub-second accuracy rounded to two decimal places. The `fps` parameter determines the temporal spacing between consecutive frames, while `offset` accounts for any starting position adjustments.

### Parsing Timestamps from Scene-Change Detection

When using scene-change extraction via `extract_scene_candidates()`, Claude Video leverages ffmpeg's `showinfo` filter to capture exact presentation timestamps. The system parses ffmpeg's output using the regular expression:

```python
SHOWINFO_TS_RE = re.compile(r"pts_time:([0-9.]+)")

```

This regex extracts `pts_time` values from ffmpeg's filter output, creating timestamps that align exactly with the video's internal clock rather than calculated approximations. The extracted values correspond precisely to the saved JPEG files, ensuring temporal accuracy for scene-based frames.

## Sorting Frame Files by Physical Sequence

Before timestamp assignment, Claude Video enforces chronological order by sorting physical files on disk. Both extraction methods sort the output directory contents using:

```python
sorted(out_dir.glob("frame_*.jpg"))

```

This step guarantees that the physical order of JPEG files matches the chronological order in which ffmpeg wrote them. By sorting on disk before processing timestamps, the system creates a deterministic sequence that corresponds to the source video's temporal flow, preventing filesystem ordering issues from disrupting the frame sequence.

## Merging Frame Sources with Timestamp-Based Re-indexing

When combining frames from multiple sources—such as primary extraction frames and transcript cue frames—Claude Video uses the `merge_frames()` function to maintain chronological integrity. The merging process in [`skills/watch/scripts/frames.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/frames.py) re-indexes all frames by sorting on the `timestamp_seconds` key:

```python
merged = sorted([*primary, *pinned], key=lambda f: f["timestamp_seconds"])

```

This consolidation step guarantees a single, chronological sequence even when mixing different extraction engines or frame sources. The timestamp-based sort ensures that scene-change frames, uniform extraction frames, and transcript cue frames interleave correctly according to their actual temporal positions in the source video.

## Practical Implementation Examples

### Uniform Extraction with Calculated Timestamps

```python
from pathlib import Path
from skills.watch.scripts.frames import extract, get_metadata, auto_fps

video = "sample.mp4"
out_dir = Path("frames")
meta = get_metadata(video)
fps, _ = auto_fps(meta["duration_seconds"])
frames = extract(video, out_dir, fps=fps)

# `frames` is a list of dicts sorted by timestamp_seconds

print(frames[0]["timestamp_seconds"])  # → 0.00 (first frame)

print(frames[-1]["timestamp_seconds"]) # → video duration (last frame)

```

### Scene-Change Extraction with Parsed Timestamps

```python
from pathlib import Path
from skills.watch.scripts.frames import extract_scene_candidates

video = "sample.mp4"
out_dir = Path("scene_frames")
candidates = extract_scene_candidates(video, out_dir)

# Each candidate already contains an accurate timestamp

for f in candidates[:3]:
    print(f["timestamp_seconds"], f["reason"])

# → 0.00 first-frame

# → 12.34 scene-change

# → 24.78 scene-change

```

### Merging Primary and Transcript Cue Frames

```python
from skills.watch.scripts.frames import merge_frames

primary = [...]          # frames from uniform or scene extraction

pinned = [...]           # frames generated from transcript timestamps

merged = merge_frames(primary, pinned)

# The merged list is chronological:

assert merged[0]["timestamp_seconds"] <= merged[1]["timestamp_seconds"]

```

## Summary

- **Timestamp calculation** in uniform extraction uses frame index and FPS to determine temporal position with `round(offset + (i / fps), 2)` precision.
- **Scene-change extraction** parses exact `pts_time` values from ffmpeg's `showinfo` filter using regex matching for sub-second accuracy.
- **Physical file sorting** via `sorted(out_dir.glob("frame_*.jpg"))` ensures filesystem order matches chronological sequence before processing.
- **Timestamp-based merging** via `merge_frames()` re-indexes mixed frame sources using `key=lambda f: f["timestamp_seconds"]` to maintain temporal integrity across extraction methods.

## Frequently Asked Questions

### How does Claude Video handle timestamp accuracy when extracting frames at different FPS rates?

Claude Video calculates timestamps dynamically using the formula `round(offset + (i / fps if fps > 0 else 0.0), 2)`, where `i` represents the frame index and `fps` the extraction rate. This ensures that regardless of the chosen frame rate, each timestamp accurately reflects its temporal position relative to the video start, rounded to two decimal places for consistency.

### What is the difference between uniform extraction and scene-change extraction timestamps?

Uniform extraction computes timestamps mathematically from frame indices and FPS values, while scene-change extraction parses actual presentation timestamps (`pts_time`) from ffmpeg's internal clock. The latter method captures exact temporal positions where scene changes occur, providing higher precision for keyframe identification compared to calculated approximations.

### Why does Claude Video sort frames both by filename and by timestamp?

The system first sorts physical files using `sorted(out_dir.glob("frame_*.jpg"))` to ensure filesystem order matches ffmpeg's output sequence, then uses timestamp values for merging operations. This dual approach prevents filesystem metadata inconsistencies from disrupting chronological order while enabling accurate interleaving of frames from different extraction sources.

### How does the merge_frames function ensure chronological order when combining different frame sources?

The `merge_frames()` function combines primary and pinned frame lists, then re-sorts the unified collection using `key=lambda f: f["timestamp_seconds"]`. This timestamp-based re-indexing ensures that frames from uniform extraction, scene-change detection, and transcript cues interleave correctly according to their actual temporal positions in the source video, regardless of extraction method or original collection order.