# Roadmap for Future Development of video-use: 8 Milestones for the Transcript-First Video Editor

> Explore the video-use roadmap: 8 milestones for transcript-first video editing, expanding animation, enabling collaboration, and maintaining core principles. Discover the future of video creation.

- Repository: [Browser Use/video-use](https://github.com/browser-use/video-use)
- Tags: roadmap
- Published: 2026-07-10

---

**The video-use roadmap prioritizes eight concrete milestones—ranging from animation source expansion to collaborative editing—that extend the existing transcript-first architecture while maintaining zero-presets and low-token design principles.**

The `video-use` repository by browser-use implements a lightweight, transcript-first pipeline that enables language models to edit videos without loading raw frames. Understanding the **roadmap for future development of video-use** is essential for contributors and users who want to leverage its audio-first, self-evaluation architecture. The project maintains a strict separation between transcription, LLM reasoning, and rendering phases, with each milestone extending these layers while preserving the system's token efficiency.

## Current Architecture and Design Principles

The video-use core architecture consists of three distinct layers that dictate the roadmap direction. First, **audio transcription** via ElevenLabs Scribe converts speech to timestamped text. Second, **on-demand visual composites** generated by [`timeline_view.py`](https://github.com/browser-use/video-use/blob/main/timeline_view.py) provide frame context without loading full video files. Third, an **LLM-driven decision loop** produces an edit decision list (EDL) that [`render.py`](https://github.com/browser-use/video-use/blob/main/render.py) executes before final output.

This design enforces **text + on-demand visuals**, **audio-first cuts**, and a **self-evaluation loop** that validates each cut before committing it. Consequently, the roadmap focuses on extending these pillars rather than replacing them, ensuring the system remains zero-presets and production-ready according to the "12 hard rules" defined in [`SKILL.md`](https://github.com/browser-use/video-use/blob/main/SKILL.md).

## The 8-Milestone Development Roadmap

### 1. Expand Animation Sources

The first milestone adds native support for **FFmpeg-based overlays**, **Lottie**, and **Web-GPU canvases**. Rather than introducing new rendering engines, these formats will integrate into the existing [`helpers/render.py`](https://github.com/browser-use/video-use/blob/main/helpers/render.py) pipeline, splicing new overlay streams using the same injection method currently employed for HyperFrames, Remotion, and Manim. New modules `helpers/animation_*.py` will handle format-specific parsing while [`render.py`](https://github.com/browser-use/video-use/blob/main/render.py) maintains the final assembly logic.

### 2. UI / Web Front-End

A lightweight web interface will wrap the existing CLI tools, enabling browser-based footage uploads and cut confirmation. This implementation introduces a new [`app.py`](https://github.com/browser-use/video-use/blob/main/app.py) file providing a Flask or FastAPI wrapper that calls [`transcribe.py`](https://github.com/browser-use/video-use/blob/main/transcribe.py) and [`timeline_view.py`](https://github.com/browser-use/video-use/blob/main/timeline_view.py) through HTTP endpoints. The UI preserves the "LLM-only" decision flow by displaying the same self-evaluation previews that the command-line tool generates.

### 3. Cloud-Native Deployments

To support SaaS adoption, the roadmap includes a one-click Docker image and serverless "Edit-as-a-Service" endpoints. This involves containerizing the current `uv sync` environment and exposing CLI entry-points through containerized APIs. The `Dockerfile` and [`docker-compose.yml`](https://github.com/browser-use/video-use/blob/main/docker-compose.yml) will replicate the local self-evaluation loop in cloud environments without architectural changes.

### 4. Pluggable Transcriber Back-Ends

Future releases will abstract transcription behind a `Transcriber` interface, allowing **Whisper**, **AssemblyAI**, or custom ASR models to replace ElevenLabs Scribe. The [`helpers/transcribe.py`](https://github.com/browser-use/video-use/blob/main/helpers/transcribe.py) refactor will introduce a `TranscriberBase` class while keeping the `pack_transcripts` function signature stable. This ensures downstream LLM reasoning and rendering logic remains unchanged regardless of the transcription provider.

### 5. Multi-Language and Localization

Non-English footage support requires extending the transcript schema to include language tags and locale-aware subtitle presets. [`helpers/transcribe.py`](https://github.com/browser-use/video-use/blob/main/helpers/transcribe.py) will detect language codes, and [`helpers/timeline_view.py`](https://github.com/browser-use/video-use/blob/main/helpers/timeline_view.py) will render localized text overlays. The existing word-level timestamp format provides the foundation for this extension without breaking the current timeline visualization logic.

### 6. Collaborative Editing

Multiple agents and human reviewers will coordinate through a shared [`project.md`](https://github.com/browser-use/video-use/blob/main/project.md) lock-file and a "review-queue" system. A new [`helpers/collab.py`](https://github.com/browser-use/video-use/blob/main/helpers/collab.py) module will manage concurrency, while [`helpers/grade.py`](https://github.com/browser-use/video-use/blob/main/helpers/grade.py) checks the review queue before allowing the LLM to proceed past step four of the pipeline. This enables parallel cut suggestions without corrupting the EDL state.

### 7. Advanced Self-Evaluation Metrics

The self-evaluation loop will incorporate **visual similarity (SSIM)**, **audio continuity (spectral flux)**, and **semantic consistency checks**. These validators will hook into [`helpers/grade.py`](https://github.com/browser-use/video-use/blob/main/helpers/grade.py), extending the existing validation that currently runs `timeline_view` at every cut. The metrics provide quantitative guarantees for the "12 hard rules" while maintaining the asynchronous evaluation pattern.

### 8. Documentation and Community SDK

The final milestone releases an extensible Python SDK on PyPI, complete with type hints and example notebooks. Users will import `video_use` directly rather than relying on CLI invocations. Updates to [`pyproject.toml`](https://github.com/browser-use/video-use/blob/main/pyproject.toml) will define the package metadata, while the existing [`README.md`](https://github.com/browser-use/video-use/blob/main/README.md) structure provides the tutorial foundation.

## Code Implementation Examples

The following snippets demonstrate how current workflows operate and how roadmap features will integrate.

### Existing Workflow: Transcribe, Edit, and Render

```python
from video_use.helpers import transcribe, render

# Step 1: Transcribe all source files using ElevenLabs Scribe

transcribe.transcribe_folder("raw_videos")

# Step 2: LLM produces EDL (executed automatically via Claude Code or similar)

# Step 3: Render final edit with overlays and color grades

render.render_final("raw_videos")

```

### Future: Pluggable Whisper Transcriber

```python
from video_use.helpers.transcribe import WhisperTranscriber, pack_transcripts

# Initialize alternative transcriber

transcriber = WhisperTranscriber(model="large-v2")
transcribe_folder = transcriber.transcribe_folder

# Execute same pipeline with different back-end

transcribe_folder("raw_videos")
pack_transcripts("raw_videos")

```

### Future: Lottie Animation Overlays

```python
from video_use.helpers.animation_lottie import LottieOverlay
from video_use.helpers import render

# Create overlay from JSON animation file

overlay = LottieOverlay("assets/intro.json", start=0, duration=5)
render.add_overlay(overlay)
render.render_final("raw_videos")

```

## Key Files in the video-use Repository

- **[`helpers/transcribe.py`](https://github.com/browser-use/video-use/blob/main/helpers/transcribe.py)**: Wrapper around ElevenLabs Scribe; primary entry point for audio-to-text conversion. Will host the `TranscriberBase` abstraction in milestone 4.

- **[`helpers/timeline_view.py`](https://github.com/browser-use/video-use/blob/main/helpers/timeline_view.py)**: Generates on-demand visual composites (filmstrip + waveform) used for the self-evaluation loop. Handles frame sampling without full video loading.

- **[`helpers/render.py`](https://github.com/browser-use/video-use/blob/main/helpers/render.py)**: Core rendering pipeline that assembles cuts, applies color grades, and merges animation overlays. Extended by milestone 1 for Lottie and Web-GPU support.

- **[`helpers/grade.py`](https://github.com/browser-use/video-use/blob/main/helpers/grade.py)**: Performs self-evaluation checks after each render step. Will host SSIM and spectral flux validators in milestone 7, and check the collaborative review queue in milestone 6.

- **[`SKILL.md`](https://github.com/browser-use/video-use/blob/main/SKILL.md)**: Defines the "12 hard rules" and production guidelines that constrain the LLM's decision-making process.

- **[`README.md`](https://github.com/browser-use/video-use/blob/main/README.md)**: High-level overview of the transcript-first pipeline and design philosophy.

## Summary

- The video-use roadmap extends the existing three-layer architecture (transcription → LLM reasoning → rendering) without breaking changes.
- **Animation expansion** leverages [`render.py`](https://github.com/browser-use/video-use/blob/main/render.py) to support Lottie and FFmpeg overlays.
- **Cloud and UI milestones** containerize the current CLI workflow via [`app.py`](https://github.com/browser-use/video-use/blob/main/app.py) and Docker configurations.
- **Pluggable transcribers** require a `TranscriberBase` interface in [`transcribe.py`](https://github.com/browser-use/video-use/blob/main/transcribe.py) to support Whisper and AssemblyAI.
- **Collaborative features** introduce [`collab.py`](https://github.com/browser-use/video-use/blob/main/collab.py) and a [`project.md`](https://github.com/browser-use/video-use/blob/main/project.md) lock-file for multi-agent editing.
- All milestones preserve the zero-presets, low-token, and self-evaluation principles established in the current codebase.

## Frequently Asked Questions

### What is the video-use transcript-first pipeline?

The transcript-first pipeline is an architecture where language models edit videos by reasoning over text transcripts and on-demand visual composites rather than raw frame data. According to the browser-use/video-use source code, this approach uses [`helpers/transcribe.py`](https://github.com/browser-use/video-use/blob/main/helpers/transcribe.py) for audio conversion, [`helpers/timeline_view.py`](https://github.com/browser-use/video-use/blob/main/helpers/timeline_view.py) for lightweight visual context, and an LLM loop to generate an edit decision list before [`helpers/render.py`](https://github.com/browser-use/video-use/blob/main/helpers/render.py) produces the final output.

### How does the roadmap maintain zero-presets design?

Each milestone extends specific pipeline stages without introducing default configurations. For example, animation sources (milestone 1) inject into the existing [`render.py`](https://github.com/browser-use/video-use/blob/main/render.py) stream without preset templates, while pluggable transcribers (milestone 4) abstract the audio input stage without changing the LLM reasoning logic. The [`SKILL.md`](https://github.com/browser-use/video-use/blob/main/SKILL.md) file enforces the "12 hard rules" that prevent hard-coded presets across all new features.

### Will video-use support real-time collaborative editing?

Milestone 6 introduces collaborative editing through a shared [`project.md`](https://github.com/browser-use/video-use/blob/main/project.md) lock-file and a review queue managed by [`helpers/collab.py`](https://github.com/browser-use/video-use/blob/main/helpers/collab.py). While not real-time in the traditional sense of simultaneous frame editing, this system allows multiple agents or human reviewers to suggest and approve cuts asynchronously before the LLM proceeds to the next pipeline step.

### Can I use custom transcription models with video-use?

Yes. The roadmap explicitly includes a `TranscriberBase` interface in [`helpers/transcribe.py`](https://github.com/browser-use/video-use/blob/main/helpers/transcribe.py) that allows you to implement custom ASR models such as Whisper or AssemblyAI. You instantiate your custom transcriber, call `transcribe_folder`, and the rest of the pipeline—including `pack_transcripts` and `render_final`—functions identically to the default ElevenLabs Scribe implementation.