# How ego-lite's Video and Screencast Recording Feature Works: A Deep Dive into the VideoRecorder Pipeline

> Discover how ego-lite's VideoRecorder pipeline streams CDP image frames via FFmpeg to create WebM video files for browser session recording. Learn the technical details.

- Repository: [CitroLabs/ego-lite](https://github.com/citrolabs/ego-lite)
- Tags: deep-dive
- Published: 2026-08-01

---

**Ego-lite's video and screencast recording feature captures browser sessions by streaming CDP-acquired image frames through an FFmpeg pipeline to produce WebM video files.**

The **ego-lite** browser automation framework provides agents with native video recording capabilities through its `VideoRecorder` class and `recordScreen` helper. This system bridges Chrome DevTools Protocol (CDP) frame capture with FFmpeg encoding, enabling high-quality screencast generation without external dependencies beyond FFmpeg itself.

## Core Architecture: From Browser Pixels to WebM Files

The recording pipeline spans three architectural layers in the `citrolabs/ego-lite` codebase.

### Frame Acquisition via CDP

The **screencast driver** in [`src/driver/screencast.ts`](https://github.com/citrolabs/ego-lite/blob/main/src/driver/screencast.ts) orchestrates raw frame capture from the browser:

- Connects to the browser's CDP session to receive bitmap data
- Subscribes to screencast events or explicitly calls `Page.captureScreenshot`
- Attaches millisecond-precision timestamps to each frame buffer
- Forwards frames to the recorder via `writeFrame(buffer, timestamp)`

This driver runs continuously during recording, ensuring no visual state changes are lost regardless of page animation or user interaction.

### FFmpeg Encoding Pipeline

The `VideoRecorder` class in [`src/video-recorder.ts`](https://github.com/citrolabs/ego-lite/blob/main/src/video-recorder.ts) manages the actual video encoding. When `start()` is invoked, it spawns an FFmpeg child process with these configured parameters:

```bash
ffmpeg -i pipe:0
  -framerate 25
  -c:v vp8
  -qmin 0 -qmax 50 -crf 8
  -deadline realtime -speed 4
  -threads 1
  -vf "scale=1280:720:force_original_aspect_ratio=decrease,pad=1280:720:-1:-1:color=black"
  output.webm

```

Key encoding decisions implemented in ego-lite:

- **MJPEG input**: Raw frames arrive as JPEG buffers via stdin pipe
- **VP8 codec**: Selected for broad browser compatibility and efficient compression
- **CRF 8**: Balanced quality setting that preserves text legibility
- **Single-threaded encoding**: Prevents FFmpeg from overwhelming the host CPU during concurrent browser operations

The FFmpeg binary path is resolved through the `EGO_BROWSER_FFMPEG_PATH` environment variable, falling back to system `PATH` resolution.

### Temporal Frame Handling

Ego-lite maintains accurate video timing through frame queuing logic in `_queueFrames()`:

1. Computes expected frame index from `(timestamp - firstTimestamp) / frameDurationMs`
2. Detects gaps where frames arrived slower than the 25 FPS target
3. Duplicates the prior frame to fill missing slots, preventing video desync
4. Enforces monotonic ordering before feeding FFmpeg

This compensation handles real-world conditions like network latency, JavaScript execution pauses, or CDP backpressure.

## Recording Lifecycle: Start, Stream, and Stop

### Initiating Recording

The high-level entry point `recordScreen()` in [`src/helpers.ts`](https://github.com/citrolabs/ego-lite/blob/main/src/helpers.ts) abstracts setup complexity:

```javascript
const { recordScreen } = globalThis.ego;

await recordScreen({
  outputPath: '/tmp/demo.webm',
  size: { width: 1280, height: 720 },
  // ffmpegPath: '/opt/ffmpeg/bin/ffmpeg' // optional override
});

```

Behind this helper, the runtime:
- Validates the output directory and creates it if needed
- Instantiates `VideoRecorder` with size-normalized options
- Activates the screencast driver with matching viewport dimensions
- Returns a control object with `stop()` method

### Manual VideoRecorder Usage

For custom tooling, import and control the recorder directly:

```javascript
import { VideoRecorder } from 'ego-browser/src/video-recorder.js';

const recorder = new VideoRecorder({
  outputPath: '/tmp/custom.webm',
  size: { width: 1024, height: 768 },
});

await recorder.start();

// Push frames from any source
const { data: pngBuffer } = await ego.sendCDPMessage(
  'Page.captureScreenshot',
  { format: 'png' }
);
recorder.writeFrame(pngBuffer, Date.now());

// Finalize
await recorder.stop();

```

### Graceful Shutdown Sequence

The `stop()` method in [`src/video-recorder.ts`](https://github.com/citrolabs/ego-lite/blob/main/src/video-recorder.ts) executes a careful teardown:

1. **Frame flush**: Empties any queued frames, duplicating the last frame to reach expected duration
2. **Stream closure**: Sends EOF to FFmpeg's stdin pipe
3. **Process await**: Waits for FFmpeg subprocess exit with timeout protection
4. **Exit validation**: Checks code; on failure, captures stderr (capped at 64KB) and cleans up temp files
5. **Atomic rename**: Moves completed recording from `.tmp.webm` to final `outputPath`

This sequence guarantees that `await recorder.stop()` resolves only when the video file is fully written and verified, enabling reliable downstream processing.

## Error Handling and Operational Safety

Ego-lite implements defensive patterns for production reliability:

| Scenario | Handling |
|----------|----------|
| **Missing FFmpeg** | Clear runtime error with installation instructions and `EGO_BROWSER_FFMPEG_PATH` documentation |
| **FFmpeg crash** | Stderr preserved (64KB limit), temp file removed, descriptive exception thrown |
| **Filesystem races** | All I/O uses `await` — `mkdir`, `rename`, `unlink` — preventing partial states |
| **Memory pressure** | Frame buffers flow through streams without accumulation; stderr buffering capped |

The 64KB stderr cap prevents runaway log accumulation from broken FFmpeg invocations while preserving diagnostic context.

## Integration with Browser Runtime

The [`src/browser-runtime.ts`](https://github.com/citrolabs/ego-lite/blob/main/src/browser-runtime.ts) component wires recording into ego-lite's broader automation system:

- Maintains CDP session state and event routing
- Forwards `Page.screencastFrame` events to registered drivers
- Handles viewport resizing to match requested recording dimensions
- Coordinates cleanup when browser contexts close unexpectedly

This integration means agents need no manual CDP knowledge — the `recordScreen` helper manages session negotiation internally.

## Summary

- **Frame source**: CDP commands in [`src/driver/screencast.ts`](https://github.com/citrolabs/ego-lite/blob/main/src/driver/screencast.ts) capture browser bitmaps with timestamps
- **Encoding engine**: `VideoRecorder` in [`src/video-recorder.ts`](https://github.com/citrolabs/ego-lite/blob/main/src/video-recorder.ts) streams MJPEG to FFmpeg, producing VP8 WebM at 25 FPS
- **Temporal accuracy**: Frame queuing logic fills gaps to maintain synchronized video output
- **Operational safety**: Async I/O, stderr limits, and atomic file operations prevent data loss
- **Environment control**: `EGO_BROWSER_FFMPEG_PATH` customizes the encoder binary location

## Frequently Asked Questions

### How do I specify a custom FFmpeg binary for ego-lite video recording?

Set the `EGO_BROWSER_FFMPEG_PATH` environment variable to the full path of your FFmpeg executable. Alternatively, pass `ffmpegPath` directly to `recordScreen()` options or the `VideoRecorder` constructor. If neither is provided, ego-lite resolves `ffmpeg` from the system `PATH`.

### What video format and quality does ego-lite produce?

Ego-lite outputs **WebM files with VP8 video** at a constant rate factor of 8, yielding high-quality recordings suitable for documentation and debugging. The fixed 25 FPS output includes padded or scaled frames to match your requested `size` dimensions exactly.

### Can I record video without using the high-level `recordScreen` helper?

Yes. Import `VideoRecorder` from [`ego-browser/src/video-recorder.js`](https://github.com/citrolabs/ego-lite/blob/main/ego-browser/src/video-recorder.js) and manage frames manually. Call `start()` to spawn FFmpeg, `writeFrame(buffer, timestamp)` for each image, and `await stop()` to finalize. This is useful for custom frame sources or post-processing pipelines.

### Why does ego-lite duplicate frames during recording?

The recorder in [`src/video-recorder.ts`](https://github.com/citrolabs/ego-lite/blob/main/src/video-recorder.ts) detects when incoming frames arrive slower than the 25 FPS target and duplicates the prior frame to fill temporal gaps. This preserves video duration accuracy and prevents playback desynchronization when CDP capture experiences latency or backpressure.