# How Screencast and Video Recording Works in Ego Lite: CDP and FFmpeg Explained

> Learn how Ego Lite's screencast and video recording works by exploring CDP and FFmpeg for efficient VP8 encoding into WebM files.

- Repository: [CitroLabs/ego-lite](https://github.com/citrolabs/ego-lite)
- Tags: internals
- Published: 2026-08-25

---

**The Ego Lite browser harness captures screen recordings by streaming raw JPEG frames over Chrome DevTools Protocol (CDP) and piping them through FFmpeg for VP8 encoding into WebM files.**

The **screencast system** in [ego-lite](https://github.com/citrolabs/ego-lite) provides automated agents with a lightweight, disposable API for recording browser interactions. Built on TypeScript and FFmpeg, it avoids heavy native dependencies while producing standard WebM output suitable for debugging, demos, and regression testing.

---

## Overview: Two-Module Architecture

The video recording functionality splits cleanly between orchestration and encoding:

- **`startScreencast` / `stopScreencast`** ([`src/driver/screencast.ts`](https://github.com/citrolabs/ego-lite/blob/main/src/driver/screencast.ts)) — manages CDP session lifecycle, frame subscription, and timing
- **`VideoRecorder`** ([`src/video-recorder.ts`](https://github.com/citrolabs/ego-lite/blob/main/src/video-recorder.ts)) — wraps FFmpeg for JPEG-to-VP8 transcoding

This separation keeps the JavaScript side deterministic while delegating CPU-intensive encoding to a battle-tested external process.

---

## The Driver Layer: Capturing Frames via CDP

Implementation file: [`src/driver/screencast.ts`](https://github.com/citrolabs/ego-lite/blob/main/src/driver/screencast.ts) ([source](https://github.com/citrolabs/ego-lite/blob/main/package/ego-browser/src/driver/screencast.ts))

### Starting a Screencast

When `startScreencast()` is invoked, it performs six sequential steps:

1. **Validates output path** — enforces `.webm` extension and parses quality settings (default: 90)
2. **Ensures CDP session** — calls `ensureSession()` and determines capture size via `defaultSize` (scales viewport down by default) at lines 58-62
3. **Spawns recorder** — creates a `VideoRecorder` instance through `dependencies.createRecorder` and starts the FFmpeg process at lines 62-66
4. **Subscribes to frames** — listens for `Page.screencastFrame` events at lines 76-92
5. **Adjusts timestamps** — rewrites frame timing to ensure monotonic progression
6. **Acknowledges frames** — sends `Page.screencastFrameAck` back to the browser to receive the next frame
7. **Starts CDP stream** — issues `Page.startScreencast` command at lines 101-112

The subscription handler at lines 76-92 is performance-critical: each incoming JPEG buffer is immediately forwarded to the recorder with an adjusted timestamp, then acknowledged without awaiting disk I/O.

### Stopping a Screencast

The `stopScreencast` function (lines 29-68) handles cleanup:

1. Unsubscribes from `Page.screencastFrame` to stop frame ingestion
2. Awaits pending frame writes to complete
3. **Fallback screenshot** — generates a single PNG if zero frames arrived (prevents empty output files)
4. Sends `Page.stopScreencast` to terminate the browser stream
5. Finalizes FFmpeg and renames the temporary file to the target path

The async-dispose pattern ensures recordings complete even when scripts exit abruptly.

---

## The Recorder Layer: FFmpeg Encoding

Implementation file: [`src/video-recorder.ts`](https://github.com/citrolabs/ego-lite/blob/main/src/video-recorder.ts) ([source](https://github.com/citrolabs/ego-lite/blob/main/package/ego-browser/src/video-recorder.ts))

The `VideoRecorder` class spans 84 lines and performs three core operations:

### 1. Process Initialization (`start()`)

Lines 44-82 build and spawn FFmpeg with these arguments:

```bash
ffmpeg -f image2pipe -i - -c:v vp8 -r 25 -deadline realtime -b:v 1M -auto-alt-ref 0 output.webm

```

- **`-f image2pipe`** — reads concatenated JPEG images from stdin
- **`-c:v vp8`** — VP8 codec for broad browser compatibility
- **`-r 25`** — hard-coded 25 fps output
- **`-deadline realtime`** — prioritizes encoding speed over compression ratio

The temporary output path avoids corrupt files if the process crashes.

### 2. Frame Writing (`writeFrame()`)

Lines 21-31 handle frame timing logic:

```typescript
writeFrame(buffer: Buffer, timestamp: number): void {
  const frameNumber = Math.floor(timestamp * 25 / 1000); // Convert ms to frame index at 25fps
  const repeats = Math.max(1, frameNumber - this.lastFrameNumber);
  // Queue buffer for 'repeats' frames to smooth dropped intervals
}

```

The 25 fps assumption means frame intervals are quantized to 40ms boundaries. If CDP delivers frames irregularly, the recorder duplicates buffers to maintain consistent playback speed.

### 3. Finalization (`stop()`)

Lines 44-85 perform graceful shutdown:

- Flushes remaining frame repeats to stdin
- Closes stdin pipe (FFmpeg interprets this as EOF)
- Awaits process exit code
- Checks stderr for encoding errors
- Cleans up temporary file on failure, or renames on success

Error handling distinguishes between **recoverable** (temporary cleanup) and **fatal** (exception throwing) conditions.

---

## Usage: High-Level and Low-Level APIs

### Page-Aware Convenience Method

```javascript
// Start a screencast that records the visible viewport into "demo.webm"
const { dispose } = await page.screencast.start({
  path: "demo.webm",      // must end with .webm
  size: { width: 800, height: 600 }, // optional; defaults to scaled viewport
  quality: 85,            // JPEG quality 0-100 (default 90)
});

// ... interact with the page, click, scroll, etc. ...

// When done, stop and finalize the recording
await dispose();   // equivalent to await page.screencast.stop()

```

The `dispose` method is **idempotent** — safe to call multiple times or leave for finalization hooks.

### Direct Driver Access

```javascript
// Using the low-level API directly (rarely needed)
import { startScreencast, stopScreencast } from "ego-browser/src/driver/screencast.js";

await startScreencast({ path: "out.webm" });
// ... perform actions …
await stopScreencast();   // resolves after FFmpeg finishes writing the file

```

Low-level access bypasses page object lifecycle management but requires manual session coordination.

---

## Configuration Reference

| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `path` | `string` | *(required)* | Output file path; must use `.webm` extension |
| `size.width` | `number` | scaled viewport | Capture width in pixels |
| `size.height` | `number` | scaled viewport | Capture height in pixels |
| `quality` | `number` | 90 | JPEG compression quality (0-100) |

---

## Key Source Files and Tests

| File | Purpose |
|------|---------|
| [`src/driver/screencast.ts`](https://github.com/citrolabs/ego-lite/blob/main/src/driver/screencast.ts) | CDP orchestration and public API |
| [`src/video-recorder.ts`](https://github.com/citrolabs/ego-lite/blob/main/src/video-recorder.ts) | FFmpeg process management |
| `src/driver/screencast.test.mjs` | Unit tests for start/stop logic and error paths |
| `scripts/real-browser-e2e/cases/video-recording.mjs` | Full integration example |

---

## Summary

- **CDP integration** — `Page.screencastFrame` events deliver raw JPEG buffers from the browser engine
- **FFmpeg encoding** — `VideoRecorder` pipes frames through `image2pipe` to produce VP8 WebM at 25 fps
- **Disposable API** — `startScreencast` returns an async-dispose object for reliable cleanup
- **Fault tolerance** — fallback screenshot prevents empty outputs; temporary files protect against crashes
- **Minimal dependencies** — requires only a system FFmpeg installation; no native Node.js modules

---

## Frequently Asked Questions

### What video format does Ego Lite produce?

Ego Lite outputs **VP8-encoded WebM files** with silent audio tracks. This format was chosen for universal browser support, reasonable compression, and FFmpeg's mature VP8 encoder. The hard-coded 25 fps and realtime deadline prioritize encoding speed over file size.

### Why does the recorder duplicate frames?

The `writeFrame()` method in [`src/video-recorder.ts`](https://github.com/citrolabs/ego-lite/blob/main/src/video-recorder.ts) calculates frame numbers based on millisecond timestamps and a fixed 25 fps clock. When CDP delivers frames at irregular intervals, the recorder **repeats buffers** to fill gaps, producing smooth playback even under load. This trade-off accepts minor quality degradation for timing consistency.

### Can I change the frame rate or codec?

Not without modifying [`src/video-recorder.ts`](https://github.com/citrolabs/ego-lite/blob/main/src/video-recorder.ts). The 25 fps assumption appears in three places: FFmpeg arguments (`-r 25`), the timestamp-to-frame conversion (`timestamp * 25 / 1000`), and frame repeat logic. Codec selection (`-c:v vp8`) is similarly hard-coded. Pull requests to expose these parameters would need to synchronize timing math across the recorder class.

### What happens if no frames arrive before stopping?

The stop handler in [`src/driver/screencast.ts`](https://github.com/citrolabs/ego-lite/blob/main/src/driver/screencast.ts) detects zero-frame recordings and **falls back to a single screenshot**. This prevents empty WebM files that many players reject. The screenshot uses the same size and quality parameters as the attempted recording, providing diagnostic value even when CDP streaming fails.