How the EDL JSON Format Defines Video Cuts, Overlays, and Subtitles in video-use

The EDL JSON format provides a declarative schema that maps source media identifiers to specific time ranges, color grades, animated overlays, and subtitle tracks, enabling automated video assembly via ffmpeg.

The browser-use/video-use repository orchestrates complex video editing workflows through a single machine-readable declaration. By leveraging the EDL JSON format, developers can specify precise cuts, layer visual effects, and burn in subtitles without manual timeline editing, as the schema drives the entire rendering pipeline defined in helpers/render.py.

Core Schema Components

The EDL JSON structure is formally specified in SKILL.md and consumed by the rendering engine. Each field controls a specific aspect of the final composite.

Version and Source Mapping

The version field (integer, currently 1) enables the renderer to detect schema changes and raise errors on incompatible files. The sources object creates a lookup table mapping short identifiers (e.g., "C0103") to absolute file system paths. During initialization, helpers/render.py uses the resolve_path function to convert these identifiers into valid media paths for ffmpeg processing.

Ranges (Video Cuts)

The ranges array defines every segment to include in the final edit. Each object requires three fields:

• source – The identifier key matching an entry in sources
• start – Start time in seconds (float)
• end – End time in seconds (float)

Optional metadata fields like beat, quote, and reason provide LLM-readable context for why the cut was selected. The extract_all_segments function in helpers/render.py loops over edl["ranges"], calling ffmpeg to extract each range as seg_XX_<source>.mp4 with optional color grading applied during extraction.

Color Grading

The grade field accepts three possible values:

• Preset name – Strings like "warm_cinematic" map to predefined ffmpeg filter chains
• Raw filter string – Direct ffmpeg syntax for custom grading
• "auto" – Triggers per-segment analysis via auto_grade_for_clip

The resolve_grade_filter function in helpers/render.py processes this value, determining which filter graph to apply during the extraction phase.

Overlay Specifications

The overlays array places animated elements above the base video. Each overlay object requires:

• file – Path to the rendered animation (e.g., edit/animations/slot_1/render.mp4)
• start_in_output – Timestamp when the overlay appears in the final timeline
• duration – Visibility length in seconds

In build_final_composite, the renderer shifts each overlay's presentation timestamp using setpts=PTS-STARTPTS+<t>/TB, then composites it onto the base video with overlay=enable='between(t,<t>,<end>)'.

Subtitle Integration

The optional subtitles field specifies a path to a master SRT file, typically generated by build_master_srt. According to the rendering rules in helpers/render.py, subtitles are applied after all overlays to ensure text remains visible and is not obscured by graphic elements. The final filtergraph appends subtitles='...' to burn the text into the video.

Duration Validation

The total_duration_s float declares the expected runtime of the final video. After concatenation, the CLI compares this value against the actual rendered duration, raising a warning if mismatches occur.

Rendering Pipeline Sequence

The helpers/render.py engine processes the EDL JSON through a strict sequential pipeline:

1. Extract – extract_all_segments cuts and grades individual clips
2. Concatenate – concat_segments losslessly joins clips into concat.mp4
3. Composite – build_final_composite layers overlays with PTS shifting
4. Subtitle – Burns the master SRT after overlay compositing
5. Normalize – Applies loudness normalization to the final output

This ordering guarantees that subtitles appear above overlays and prevents double-encoding of base video segments.

Complete EDL JSON Example

The following excerpt from SKILL.md demonstrates a fully configured edit decision list:

{
  "version": 1,
  "sources": {
    "C0103": "/abs/path/C0103.MP4",
    "C0108": "/abs/path/C0108.MP4"
  },
  "ranges": [
    {
      "source": "C0103",
      "start": 2.42,
      "end": 6.85,
      "beat": "HOOK",
      "quote": "...",
      "reason": "Cleanest delivery, stops before slip at 38.46."
    },
    {
      "source": "C0108",
      "start": 14.30,
      "end": 28.90,
      "beat": "SOLUTION",
      "quote": "...",
      "reason": "Only take without the false start."
    }
  ],
  "grade": "warm_cinematic",
  "overlays": [
    {
      "file": "edit/animations/slot_1/render.mp4",
      "start_in_output": 0.0,
      "duration": 5.0
    }
  ],
  "subtitles": "edit/master.srt",
  "total_duration_s": 87.4
}

Implementing the Rendering Pipeline

The following Python workflow demonstrates how the EDL JSON drives the rendering process:

import json
from pathlib import Path
from helpers.render import (
    extract_all_segments,
    concat_segments,
    build_master_srt,
    build_final_composite,
)

edl_path = Path("edit/edl.json")
edl = json.loads(edl_path.read_text())

# 1️⃣ Extract per‑segment clips (with optional auto‑grading)

segments = extract_all_segments(edl, edit_dir=Path("edit"), preview=False)

# 2️⃣ Concatenate the clipped segments losslessly

concat_path = Path("edit/concat.mp4")
concat_segments(segments, concat_path, edit_dir=Path("edit"))

# 3️⃣ Generate subtitles (optional)

srt_path = Path("edit/master.srt")
build_master_srt(edl, edit_dir=Path("edit"), out_path=srt_path)

# 4️⃣ Composite overlays + subtitles onto the concatenated base

final_path = Path("edit/final.mp4")
build_final_composite(
    base_path=concat_path,
    overlays=edl.get("overlays", []),
    subtitles_path=srt_path,
    out_path=final_path,
    edit_dir=Path("edit"),
)

Summary

• The EDL JSON format in browser-use/video-use provides a complete declarative specification for automated video editing.
• Video cuts are defined in the ranges array with precise start/end timestamps and optional metadata.
• Color grading supports presets, custom ffmpeg filters, or automatic per-segment analysis via the grade field.
• Overlays are positioned using timeline coordinates and composited with PTS shifting to ensure accurate placement.
• Subtitles are burned in after overlays to maintain visibility, sourced from an SRT path specified in the JSON.
• The rendering pipeline in helpers/render.py enforces strict ordering: extract, concat, overlay, subtitle, normalize.

Frequently Asked Questions

What is the EDL JSON format in video-use?

The EDL JSON format is a machine-readable Edit Decision List schema that declares which source media to use, which time ranges to extract, how to color grade them, and where to place overlays and subtitles. It serves as the single source of truth for the automated rendering pipeline in browser-use/video-use.

How does video-use handle color grading in the EDL JSON?

The grade field accepts either a preset name (like "warm_cinematic"), a raw ffmpeg filter string, or the literal "auto". When set to "auto", the auto_grade_for_clip function analyzes each segment individually during the extraction phase to determine optimal color correction.

Can I use custom ffmpeg filters in the EDL JSON format?

Yes. The grade field supports raw ffmpeg filter syntax, allowing you to pass complex filter graphs directly to the renderer. The resolve_grade_filter function in helpers/render.py passes these strings unmodified to the ffmpeg command during segment extraction.

How are subtitles positioned relative to overlays in video-use?

Subtitles are always rendered after overlays in the filtergraph pipeline. This ensures that text remains visible and appears on top of any animated graphics, following the strict ordering rule implemented in build_final_composite where the subtitles filter is appended after all overlay operations complete.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}