# How to Interpret Frame Extraction Metadata (Engine, Candidates, Deduped Count) for Debugging in claude-video

> Debug claude-video frame extraction using metadata. Understand candidate count, deduped count, and selected count to fix frame extraction issues.

- Repository: [bradautomates/claude-video](https://github.com/bradautomates/claude-video)
- Tags: how-to-guide
- Published: 2026-08-03

---

**Extract three key numbers from claude-video's metadata JSON—`candidate_count`, `deduped_count`, and `selected_count`—to diagnose why your frame extraction produced too few or too many frames.**

When you run the `watch` skill in **bradautomates/claude-video**, the frame extraction engine prints a compact metadata object that reveals exactly what happened during processing. Understanding these numbers helps you tune parameters, identify fallback behavior, and optimize your video analysis pipeline.

## What the Frame Extraction Metadata Contains

Every extraction returns a dictionary with these consistent fields:

| Key | Purpose |
|-----|---------|
| `engine` | Which strategy ran: `scene`, `keyframe`, `uniform`, or `timestamps` |
| `candidate_count` | Raw frames detected before any filtering |
| `deduped_count` | Frames removed as near-duplicates |
| `selected_count` | Final frames written to disk |
| `fallback` | `True` if the engine reverted to uniform sampling |

In [`skills/watch/scripts/frames.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/frames.py), the `extract_scene_or_uniform` function constructs this metadata at lines 146–152:

```python
return (
    final_frames,
    {
        "engine": "scene",
        "candidate_count": len(candidates),
        "deduped_count": deduped_count,
        "selected_count": len(final_frames),
        "fallback": False,
    },
)

```

## Where Each Engine Produces Metadata

### Scene-Aware Extraction with Uniform Fallback

The primary scene detection path in `extract_scene_or_uniform` populates `engine: "scene"` when ffmpeg's scene filter detects sufficient cuts. When scene detection fails to meet `SCENE_MIN_FRAMES` (8 by default), the function falls back to uniform sampling and sets `engine: "uniform"` with `fallback: True`.

### Keyframe Extraction with Uniform Fallback

Inside `extract_keyframes`, the uniform-fallback branch at lines 663–668 creates metadata with `"engine": "uniform"`. The successful keyframe path at lines 677–682 produces `"engine": "keyframe"`.

### Timestamp-Driven Cues

The `extract_at_timestamps` function returns a tuple whose second element contains `"engine": "timestamps"` at lines 84–88. This path also includes `dropped_out_of_window` when timestamps fall outside `--start/--end` bounds.

## Decoding the Four Critical Numbers

### candidate_count: Raw Detection Volume

This reveals how "rich" your video is for the chosen engine:

- **Scene**: Number of frame-change events from ffmpeg scene detection
- **Keyframe**: Count of I-frames detected
- **Uniform**: Equals your `--max-frames` budget (no real detection)
- **Timestamps**: Number of timestamps you specified

A low `candidate_count` with `fallback: True` indicates the engine couldn't find enough content.

### deduped_count: Static Content Indicator

Perceptual deduplication runs through `_dedupe_by_deltas` in `dedupe_perceptual` (lines 66–71). The default `DEDUP_THRESHOLD` of 2.0 collapses frames with similar thumbnails.

High `deduped_count` values suggest:
- Screen recordings with static slides
- Long stationary shots
- Video with minimal motion

To retain all frames, use `--no-dedup`.

### selected_count: Final Output Size

This results from three processing stages:
1. Deduplication (if enabled)
2. Even sampling to respect budget caps via `_even_sample`
3. Fallback triggers when minimum thresholds aren't met

When `selected_count` equals `candidate_count - deduped_count`, your video hit no budget constraints.

### fallback: Engine Behavior Flag

`True` means the primary engine failed its minimum requirements:
- Scene detection found fewer than `SCENE_MIN_FRAMES` cuts
- Keyframe detection found fewer than `KEYFRAME_MIN` I-frames

The system automatically reverted to uniform sampling to ensure usable output.

## Common Debugging Scenarios

| Symptom | Metadata Signature | Fix |
|---------|-------------------|-----|
| Far fewer frames than expected | `selected_count` << `candidate_count`, `fallback: True` | Increase `--max-frames` or use richer engine (`--detail balanced`) |
| Massive deduplication | `deduped_count` ≈ `candidate_count` | Accept reduction for static content, or use `--no-dedup` |
| Zero candidates | `candidate_count: 0` | Check `--timestamps` against `--start/--end` window |
| Wrong engine active | `engine: "uniform"` despite scene/keyframe request | Verify video has enough cuts/keyframes; fallback is working correctly |

## Practical Debugging Examples

### Direct frames.py execution

```bash
python -m skills.watch.scripts.frames \
    /path/to/video.mp4 /tmp/frames \
    --fps 1.5 --resolution 640 --max-frames 50

```

Output metadata:

```json
{
  "engine": "uniform",
  "candidate_count": 50,
  "deduped_count": 12,
  "selected_count": 38,
  "fallback": false
}

```

Interpretation: 38 distinct frames from 50 candidates after removing 12 duplicates.

### High-level watch command with report

```bash
python -m skills.watch.scripts.watch \
    "https://youtu.be/abc123" \
    --detail balanced --max-frames 80

```

Report line:

```

- **Frames:** 62 selected from 94 candidates (scene, 7 near-duplicates dropped, full range, budget 80, cap 80)

```

Breakdown:
- `94` = `candidate_count`
- `7` = `deduped_count`
- `62` = `selected_count`

"with uniform fallback" would indicate `fallback: True`.

### Timestamp extraction debugging

```bash
python -m skills.watch.scripts.watch \
    video.mp4 \
    --timestamps "00:10,00:45,01:30" \
    --detail transcript

```

If timestamps fall outside your range window, check for `dropped_out_of_window` in the raw metadata from `extract_at_timestamps`.

## Key Source Files for Deep Debugging

| File | Purpose | Critical Lines |
|------|---------|--------------|
| [`skills/watch/scripts/frames.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/frames.py) | Metadata construction, all engines | Scene: 146–152; Keyframe: 663–668, 677–682; Timestamps: 84–88 |
| [`skills/watch/scripts/watch.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/watch.py) | Report generation from metadata | 96–100 |
| [`skills/watch/scripts/config.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/config.py) | Default caps and thresholds | `frame_cap`, `SCENE_MIN_FRAMES`, `KEYFRAME_MIN` |

To adjust behavior, modify `DEDUP_THRESHOLD` (2.0), `SCENE_THRESHOLD`, or minimum frame constants directly in [`frames.py`](https://github.com/bradautomates/claude-video/blob/main/frames.py) or override via CLI flags.

## Summary

- **Read the metadata JSON** that every claude-video extraction returns to understand engine behavior
- **Compare `candidate_count` to `selected_count`** to detect budget constraints or aggressive deduplication
- **Watch for `fallback: True`** which signals automatic engine switching when primary methods fail
- **Use `--no-dedup`, `--max-frames`, and `--detail`** to control the three levers: duplicate removal, output volume, and extraction strategy

## Frequently Asked Questions

### Why does my scene extraction show `engine: "uniform"` with `fallback: true`?

The scene detector found fewer than 8 cuts (`SCENE_MIN_FRAMES`), so it fell back to uniform sampling. Increase your frame budget with `--max-frames` or accept the fallback for videos with minimal scene changes.

### How do I completely disable deduplication to see all candidate frames?

Add `--no-dedup` to your `watch` command. This sets `deduped_count` to 0 and ensures `selected_count` reflects only budget capping, not perceptual similarity filtering.

### What's the difference between `candidate_count` being low versus `deduped_count` being high?

Low `candidate_count` means the engine found little inherent content (few scene cuts, sparse keyframes, or limited timestamps). High `deduped_count` means plenty of candidates existed, but many were visually similar. The first needs a different engine or video; the second is expected for static content.

### Where does the human-readable report get these numbers from?

The [`watch.py`](https://github.com/bradautomates/claude-video/blob/main/watch.py) orchestrator reads the metadata tuple returned by extraction functions and injects values into markdown at lines 96–100, formatting them into the "Frames:" line you see in output.