How ego-lite's Video and Screencast Recording Feature Works: A Deep Dive into the VideoRecorder Pipeline

Ego-lite's video and screencast recording feature captures browser sessions by streaming CDP-acquired image frames through an FFmpeg pipeline to produce WebM video files.

The ego-lite browser automation framework provides agents with native video recording capabilities through its VideoRecorder class and recordScreen helper. This system bridges Chrome DevTools Protocol (CDP) frame capture with FFmpeg encoding, enabling high-quality screencast generation without external dependencies beyond FFmpeg itself.

Core Architecture: From Browser Pixels to WebM Files

The recording pipeline spans three architectural layers in the citrolabs/ego-lite codebase.

Frame Acquisition via CDP

The screencast driver in src/driver/screencast.ts orchestrates raw frame capture from the browser:

• Connects to the browser's CDP session to receive bitmap data
• Subscribes to screencast events or explicitly calls Page.captureScreenshot
• Attaches millisecond-precision timestamps to each frame buffer
• Forwards frames to the recorder via writeFrame(buffer, timestamp)

This driver runs continuously during recording, ensuring no visual state changes are lost regardless of page animation or user interaction.

FFmpeg Encoding Pipeline

The VideoRecorder class in src/video-recorder.ts manages the actual video encoding. When start() is invoked, it spawns an FFmpeg child process with these configured parameters:

ffmpeg -i pipe:0
  -framerate 25
  -c:v vp8
  -qmin 0 -qmax 50 -crf 8
  -deadline realtime -speed 4
  -threads 1
  -vf "scale=1280:720:force_original_aspect_ratio=decrease,pad=1280:720:-1:-1:color=black"
  output.webm

Key encoding decisions implemented in ego-lite:

• MJPEG input: Raw frames arrive as JPEG buffers via stdin pipe
• VP8 codec: Selected for broad browser compatibility and efficient compression
• CRF 8: Balanced quality setting that preserves text legibility
• Single-threaded encoding: Prevents FFmpeg from overwhelming the host CPU during concurrent browser operations

The FFmpeg binary path is resolved through the EGO_BROWSER_FFMPEG_PATH environment variable, falling back to system PATH resolution.

Temporal Frame Handling

Ego-lite maintains accurate video timing through frame queuing logic in _queueFrames():

1. Computes expected frame index from (timestamp - firstTimestamp) / frameDurationMs
2. Detects gaps where frames arrived slower than the 25 FPS target
3. Duplicates the prior frame to fill missing slots, preventing video desync
4. Enforces monotonic ordering before feeding FFmpeg

This compensation handles real-world conditions like network latency, JavaScript execution pauses, or CDP backpressure.

Recording Lifecycle: Start, Stream, and Stop

Initiating Recording

The high-level entry point recordScreen() in src/helpers.ts abstracts setup complexity:

const { recordScreen } = globalThis.ego;

await recordScreen({
  outputPath: '/tmp/demo.webm',
  size: { width: 1280, height: 720 },
  // ffmpegPath: '/opt/ffmpeg/bin/ffmpeg' // optional override
});

Behind this helper, the runtime:

• Validates the output directory and creates it if needed
• Instantiates VideoRecorder with size-normalized options
• Activates the screencast driver with matching viewport dimensions
• Returns a control object with stop() method

Manual VideoRecorder Usage

For custom tooling, import and control the recorder directly:

import { VideoRecorder } from 'ego-browser/src/video-recorder.js';

const recorder = new VideoRecorder({
  outputPath: '/tmp/custom.webm',
  size: { width: 1024, height: 768 },
});

await recorder.start();

// Push frames from any source
const { data: pngBuffer } = await ego.sendCDPMessage(
  'Page.captureScreenshot',
  { format: 'png' }
);
recorder.writeFrame(pngBuffer, Date.now());

// Finalize
await recorder.stop();

Graceful Shutdown Sequence

The stop() method in src/video-recorder.ts executes a careful teardown:

1. Frame flush: Empties any queued frames, duplicating the last frame to reach expected duration
2. Stream closure: Sends EOF to FFmpeg's stdin pipe
3. Process await: Waits for FFmpeg subprocess exit with timeout protection
4. Exit validation: Checks code; on failure, captures stderr (capped at 64KB) and cleans up temp files
5. Atomic rename: Moves completed recording from .tmp.webm to final outputPath

This sequence guarantees that await recorder.stop() resolves only when the video file is fully written and verified, enabling reliable downstream processing.

Error Handling and Operational Safety

Ego-lite implements defensive patterns for production reliability:

Scenario	Handling
Missing FFmpeg	Clear runtime error with installation instructions and EGO_BROWSER_FFMPEG_PATH documentation
FFmpeg crash	Stderr preserved (64KB limit), temp file removed, descriptive exception thrown
Filesystem races	All I/O uses await — mkdir, rename, unlink — preventing partial states
Memory pressure	Frame buffers flow through streams without accumulation; stderr buffering capped

The 64KB stderr cap prevents runaway log accumulation from broken FFmpeg invocations while preserving diagnostic context.

Integration with Browser Runtime

The src/browser-runtime.ts component wires recording into ego-lite's broader automation system:

• Maintains CDP session state and event routing
• Forwards Page.screencastFrame events to registered drivers
• Handles viewport resizing to match requested recording dimensions
• Coordinates cleanup when browser contexts close unexpectedly

This integration means agents need no manual CDP knowledge — the recordScreen helper manages session negotiation internally.

Summary

• Frame source: CDP commands in src/driver/screencast.ts capture browser bitmaps with timestamps
• Encoding engine: VideoRecorder in src/video-recorder.ts streams MJPEG to FFmpeg, producing VP8 WebM at 25 FPS
• Temporal accuracy: Frame queuing logic fills gaps to maintain synchronized video output
• Operational safety: Async I/O, stderr limits, and atomic file operations prevent data loss
• Environment control: EGO_BROWSER_FFMPEG_PATH customizes the encoder binary location

Frequently Asked Questions

How do I specify a custom FFmpeg binary for ego-lite video recording?

Set the EGO_BROWSER_FFMPEG_PATH environment variable to the full path of your FFmpeg executable. Alternatively, pass ffmpegPath directly to recordScreen() options or the VideoRecorder constructor. If neither is provided, ego-lite resolves ffmpeg from the system PATH.

What video format and quality does ego-lite produce?

Ego-lite outputs WebM files with VP8 video at a constant rate factor of 8, yielding high-quality recordings suitable for documentation and debugging. The fixed 25 FPS output includes padded or scaled frames to match your requested size dimensions exactly.

Can I record video without using the high-level recordScreen helper?

Yes. Import VideoRecorder from ego-browser/src/video-recorder.js and manage frames manually. Call start() to spawn FFmpeg, writeFrame(buffer, timestamp) for each image, and await stop() to finalize. This is useful for custom frame sources or post-processing pipelines.

Why does ego-lite duplicate frames during recording?

The recorder in src/video-recorder.ts detects when incoming frames arrive slower than the 25 FPS target and duplicates the prior frame to fill temporal gaps. This preserves video duration accuracy and prevents playback desynchronization when CDP capture experiences latency or backpressure.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}