# How to Get Started with Video-Use Development: A Complete Setup Guide

> Start video use development by cloning the browser-use/video-use repo. Install dependencies and set up your LLM agent for natural language video editing via a transcription-to-render pipeline.

- Repository: [Browser Use/video-use](https://github.com/browser-use/video-use)
- Tags: getting-started
- Published: 2026-07-10

---

**Video-use development starts with cloning the browser-use/video-use repository, installing dependencies (ffmpeg and ElevenLabs API), and symlinking the skill into your LLM agent's directory to enable natural-language video editing through a seven-layer transcription-to-render pipeline.**

Video-use is an open-source tool that enables you to edit videos through natural-language interaction with Claude Code or other LLM agents. The repository implements a lightweight pipeline that keeps the LLM’s context window small by using text transcripts and on-demand visuals rather than exhaustive frame dumps. According to the browser-use/video-use source code, the architecture follows five core principles: text-plus-on-demand visuals, audio-primary editing, explicit confirmation workflows, zero content-type assumptions, and hard production rules with artistic freedom.

## Architecture Overview

The video-use development workflow is organized into seven distinct layers that transform raw footage into polished output. Each layer is implemented by specific helper modules in the repository.

### Layer 1: Audio Transcription

The pipeline begins in [`helpers/transcribe.py`](https://github.com/browser-use/video-use/blob/main/helpers/transcribe.py), which calls ElevenLabs Scribe to generate word-level transcripts including speaker diarization and audio events. The output is stored in a compact Markdown file called [`takes_packed.md`](https://github.com/browser-use/video-use/blob/main/takes_packed.md) that serves as the primary reading surface for the LLM.

### Layer 2: Visual Composite (On-Demand)

When the LLM needs visual context for ambiguous pauses or retake comparisons, [`helpers/timeline_view.py`](https://github.com/browser-use/video-use/blob/main/helpers/timeline_view.py) creates a PNG composite merging a filmstrip, waveform, and word-label overlay for the specified time range.

### Layer 3: Packing

The [`helpers/pack_transcripts.py`](https://github.com/browser-use/video-use/blob/main/helpers/pack_transcripts.py) module collates all individual transcripts into a single, tiny (~12 KB) Markdown file ([`takes_packed.md`](https://github.com/browser-use/video-use/blob/main/takes_packed.md)) that the LLM consumes.

### Layer 4: LLM Reasoning

The LLM receives the packed transcript and suggests an editing plan (cuts, color-grading, subtitles, animation overlays) based on the production rules codified in [`SKILL.md`](https://github.com/browser-use/video-use/blob/main/SKILL.md). This file contains 12 non-negotiable rules that guarantee clean output while leaving stylistic decisions to the user.

### Layer 5: Rendering

Once approved, [`helpers/render.py`](https://github.com/browser-use/video-use/blob/main/helpers/render.py) stitches the edited segments while [`helpers/grade.py`](https://github.com/browser-use/video-use/blob/main/helpers/grade.py) applies ffmpeg color-grade chains (warm, neutral, or custom). The system also handles subtitles and animation overlays via Manim, HyperFrames, Remotion, or PIL.

### Layer 6: Self-Evaluation

After each cut, the pipeline runs [`timeline_view.py`](https://github.com/browser-use/video-use/blob/main/timeline_view.py) on the freshly rendered segment. The LLM checks for visual jumps, audio pops, or missing subtitles and can trigger a re-render (maximum three attempts).

### Layer 7: Persistence

Session state—including transcripts, editing decisions, and the final EDL—is saved in [`project.md`](https://github.com/browser-use/video-use/blob/main/project.md) so you can resume work in later sessions.

## Installation and Setup

### Clone and Link the Repository

First, clone the repository and link it into your agent's skill directory:

```bash
git clone https://github.com/browser-use/video-use ~/Developer/video-use
ln -sfn ~/Developer/video-use ~/.claude/skills/video-use

# For Codex: ln -sfn ~/Developer/video-use ~/.codex/skills/video-use

```

### Install Dependencies

Install Python dependencies and system requirements:

```bash
cd ~/Developer/video-use
uv sync              # or: pip install -e .

brew install ffmpeg  # required for video processing

brew install yt-dlp  # optional, for downloading online sources

```

Configure your ElevenLabs API key:

```bash
cp .env.example .env

# Edit .env to add: ELEVENLABS_API_KEY=your_key

```

All steps are documented in [`install.md`](https://github.com/browser-use/video-use/blob/main/install.md).

## Basic Usage Workflow

### Starting a Session

Navigate to your raw footage directory and launch your LLM agent:

```bash
cd /path/to/your/raw/videos
claude  # or codex, hermes, etc.

```

Inside the LLM console, initiate editing with a natural language command:

```

> edit these into a launch video

```

### The Edit Loop

The LLM executes a five-step workflow defined in the video-use architecture:

1. **Inventory** – Scans and catalogs source files.
2. **Propose** – Generates a cutting, color-grading, subtitle, and animation plan.
3. **Confirm** – Waits for your explicit approval (`yes`/`no`) before mutating video.
4. **Render** – Executes the plan and outputs to `<videos_dir>/edit/final.mp4`.
5. **Evaluate** – Runs self-evaluation on the rendered output.

## Programmatic Python API

For custom workflows, import the helper modules directly to drive the pipeline programmatically:

```python
from helpers.transcribe import transcribe_folder
from helpers.pack_transcripts import pack_transcripts
from helpers.render import render_edl

# 1. Transcribe everything in raw/

transcribe_folder("raw/")

# 2. Pack into a single markdown view

pack_transcripts("raw/", "takes_packed.md")

# 3. Define your edit decision list (EDL)

edl = {
    "cuts": [
        {"start": 2.5, "end": 5.3, "grade": "warm"},
        {"start": 7.1, "end": 12.0, "grade": "neutral"},
    ],
    "subtitles": True,
    "animations": ["manim"],  # optional overlay types

}

# 4. Render the final output

render_edl(edl, source_dir="raw/", out_dir="edit/")

```

The LLM-generated EDL follows the format described in [`SKILL.md`](https://github.com/browser-use/video-use/blob/main/SKILL.md), making your edits reproducible.

## Summary

- Video-use development relies on a seven-layer pipeline that prioritizes audio transcripts over raw video frames to minimize LLM context usage.
- Core modules include [`helpers/transcribe.py`](https://github.com/browser-use/video-use/blob/main/helpers/transcribe.py) for audio processing, [`helpers/pack_transcripts.py`](https://github.com/browser-use/video-use/blob/main/helpers/pack_transcripts.py) for context compression, and [`helpers/render.py`](https://github.com/browser-use/video-use/blob/main/helpers/render.py) for final output.
- Installation requires symlinking the repository to `~/.claude/skills/video-use`, installing `ffmpeg`, and configuring an ElevenLabs API key.
- The workflow follows a strict ask-confirm-execute-evaluate-persist cycle codified in [`SKILL.md`](https://github.com/browser-use/video-use/blob/main/SKILL.md) with 12 non-negotiable production rules.
- You can drive the pipeline programmatically using Python functions like `transcribe_folder()`, `pack_transcripts()`, and `render_edl()`.

## Frequently Asked Questions

### What is video-use development?

Video-use development refers to building and using the video-use open-source tool that enables natural-language video editing through LLM agents. It involves setting up a seven-layer pipeline that transcribes audio, packs transcripts into lightweight Markdown, and renders final videos based on LLM-generated edit decision lists.

### Do I need Claude Code to use video-use?

No. While the repository is optimized for Claude Code with a skill directory symlink at `~/.claude/skills/video-use`, you can use other LLM agents like Codex or Hermes by adjusting the symlink path accordingly. The [`SKILL.md`](https://github.com/browser-use/video-use/blob/main/SKILL.md) file provides the production rules that any compatible agent can follow.

### What file formats does video-use support?

Video-use supports any format that ffmpeg can process, as the rendering layer relies on ffmpeg for video manipulation. The transcription layer uses ElevenLabs Scribe, which handles most common audio and video formats. Specific format limitations depend on your ffmpeg and ElevenLabs Scribe configurations.

### How does video-use keep LLM context windows small?

Instead of dumping thousands of video frames into the context window, video-use development employs text-primary transcripts stored in [`takes_packed.md`](https://github.com/browser-use/video-use/blob/main/takes_packed.md) (~12 KB per project). Visual information is provided on-demand via [`helpers/timeline_view.py`](https://github.com/browser-use/video-use/blob/main/helpers/timeline_view.py), which generates PNG composites only when the LLM needs to resolve ambiguities like retake comparisons or pause verification.