How ego-browser Handles Video Recording and Screencast Features: CDP, FFmpeg, and the VideoRecorder Class

ego-browser implements video recording and screencast features through a dedicated VideoRecorder class that wraps Chrome DevTools Protocol (CDP) commands, captures frames via Page.screencastFrame events, and optionally stitches PNG frames into MP4 format using FFmpeg.

The ego-browser library, part of the citrolabs/ego-lite repository, provides first-class support for browser automation with video capture capabilities built directly into its architecture. The video recording system combines low-level CDP integration with developer-friendly APIs for agent scripts.

Core Architecture: The VideoRecorder Class

The heart of ego-browser's screencast functionality lives in [src/video-recorder.ts](https://github.com/citrolabs/ego-lite/blob/main/package/ego-browser/src/video-recorder.ts). This class encapsulates the entire recording lifecycle while isolating CDP complexity from consumer code.

CDP Integration and Frame Capture

The VideoRecorder initiates screencasts by calling Page.startScreencast with configurable parameters:

• Frame format: PNG (default, lossless quality)
• Frame rate: 30 fps by default, customizable
• Quality: Configurable compression level

Once started, the recorder listens for Page.screencastFrame events from the Chrome DevTools Protocol. Each frame follows this handling sequence:

1. Receive base64-encoded PNG data from CDP
2. Write frame to temporary directory with sequential naming
3. Acknowledge receipt via Page.screencastFrameAck (required to maintain streaming)
4. Repeat until stop() is invoked

// From src/video-recorder.ts — conceptual flow
await this.cdpClient.send('Page.startScreencast', {
  format: 'png',
  quality: 100,
  maxWidth: 1920,
  maxHeight: 1080,
  everyNthFrame: 1
});

this.cdpClient.on('Page.screencastFrame', async (params) => {
  const { data, sessionId, metadata } = params;
  await this.writeFrame(data, metadata.timestamp);
  await this.cdpClient.send('Page.screencastFrameAck', { sessionId });
});

Video Finalization: FFmpeg Stitching or PNG Archive

When recording stops, the VideoRecorder.stop() method performs cleanup and optional format conversion:

With FFmpeg available: PNG frames are stitched into a single MP4 file using FFmpeg's concat demuxer with precise timestamp handling.

Without FFmpeg: Frames remain as a numbered PNG sequence (or ZIP archive), preserving full quality for post-processing.

// Low-level VideoRecorder usage
import { VideoRecorder } from 'ego-browser';

const recorder = new VideoRecorder({ fps: 24, outputFormat: 'mp4' });
await recorder.start();

// ... perform browser interactions ...

await recorder.stop();  // Produces recording.mp4 via FFmpeg
console.log('Video saved to:', recorder.outputPath);

High-Level API: recordVideo Helper

For most use cases, developers interact with [src/helpers.ts](https://github.com/citrolabs/ego-lite/blob/main/package/ego-browser/src/helpers.ts) and its recordVideo function. This helper abstracts instantiation, provides promise-based flow control, and returns a Readable stream for flexible consumption.

// Example: Record a navigation sequence with automatic cleanup
await egoBrowser.recordVideo(async (video) => {
  await egoBrowser.goto('https://example.com');
  await egoBrowser.waitForLoadState('networkidle');
  
  await egoBrowser.click('#start-demo');
  await egoBrowser.waitForTimeout(5000);  // Capture animation
  
  await video.stop();  // Resolves with Readable stream
}).then((stream) => {
  const fs = require('node:fs');
  const out = fs.createWriteStream('demo-recording.mp4');
  stream.pipe(out);
});

The callback pattern ensures proper resource cleanup: even if exceptions occur, the screencast session terminates correctly and temporary frames are purged.

Configuration Options and Performance Tuning

The video recording system exposes several knobs for different scenarios:

Parameter	Default	Impact
fps	30	Higher values increase smoothness but CPU/disk I/O load
format	'png'	PNG for quality; future versions may support JPEG
maxWidth/maxHeight	viewport size	Downscaling reduces file size
everyNthFrame	1	Skip frames for lower-fidelity, smaller files

For CI environments or remote agents, reducing fps to 10-15 and enabling frame skipping significantly decreases storage requirements without losing essential interaction context.

Testing and Validation

The test suite in src/video-recorder.test.mjs validates the entire pipeline:

• Mock CDP client: Simulates screencastFrame events with synthetic image data
• Frame sequence verification: Confirms correct ordering and acknowledgment
• Output validation: Verifies MP4 structure (when FFmpeg available) or PNG archive integrity

These tests ensure that frame timing, disk I/O errors, and protocol edge cases are handled gracefully.

Documentation Reference

Complete API documentation resides in [skills/ego-browser/references/video.md](https://github.com/citrolabs/ego-lite/blob/main/skills/ego-browser/references/video.md), covering:

• Required browser launch flags (--enable-automation compatibility)
• Permission handling for screen capture
• Streaming vs. file-based output patterns
• Integration with agent workspace APIs

Summary

• ego-browser video recording centers on the VideoRecorder class in src/video-recorder.ts, which manages CDP screencast lifecycle from frame capture to final output
• Chrome DevTools Protocol commands (Page.startScreencast, Page.screencastFrame, Page.stopScreencast) provide the underlying capture mechanism with explicit frame acknowledgment required
• FFmpeg integration enables MP4 output; graceful fallback to PNG sequences preserves functionality in constrained environments
• recordVideo helper in src/helpers.ts offers the primary developer API with promise-based flow control and automatic resource management
• Comprehensive test coverage in src/video-recorder.test.mjs validates frame handling, timing accuracy, and output format correctness

Frequently Asked Questions

How do I capture video without FFmpeg installed?

ego-browser automatically detects FFmpeg availability. When absent, VideoRecorder.stop() returns a path to the PNG frame directory (or zipped archive) instead of an MP4 file. You can manually convert these frames later or process them with alternative tools. Set outputFormat: 'frames' explicitly to skip FFmpeg detection.

Can I record at a specific viewport resolution?

Yes. Pass maxWidth and maxHeight in the constructor options or call egoBrowser.setViewportSize() before starting recording. The CDP screencast captures the actual rendered viewport, so ensure your browser context matches your target dimensions.

Why does my recording show dropped frames?

Dropped frames typically indicate backpressure in the acknowledgment loop. The VideoRecorder must call Page.screencastFrameAck promptly after each frame—if disk I/O blocks or the event loop stalls, Chrome pauses streaming. Reduce fps, enable everyNthFrame skipping, or use faster storage to mitigate this according to the source implementation in src/video-recorder.ts.

Is the video stream accessible during recording?

No—ego-browser's design follows a capture-then-emit pattern. The recordVideo callback receives a control object with stop() method, but the Readable stream only resolves after finalization completes. For real-time monitoring, you would need to read the temporary frame directory directly, which is unsupported and may interfere with acknowledgement timing.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}