Ego-Browser Buffered Event Queue Design: 10K Capacity CDP Event Buffering

Ego-browser implements a fixed-capacity FIFO buffer that stores up to 10,000 Chrome DevTools Protocol (CDP) events in memory, automatically truncating older entries when the limit is exceeded while providing atomic drain operations for batch retrieval.

The ego-browser package in the citrolabs/ego-lite repository manages CDP event streams through a lightweight buffered event queue designed for long-running automation agents. This implementation balances memory safety against debugging needs by capping retention at exactly 10,000 events while supporting both real-time subscriptions and batch retrieval patterns through the core browser-runtime.ts module.

Fixed-Capacity Queue Implementation in browser-runtime.ts

The buffered event queue centers on a simple JavaScript array (events) that acts as an in-memory ring buffer for CDP messages processed by the embedded ego runtime.

The 10K Event Limit (MAX_BUFFERED_EVENTS)

At lines 6–9 of browser-runtime.ts, a constant named MAX_BUFFERED_EVENTS hard-codes the capacity limit to 10,000 entries. This guard prevents unbounded memory growth when the runtime operates for extended periods, ensuring that the process footprint remains predictable regardless of session duration or CDP message velocity.

FIFO Insertion with Automatic Truncation

When the handleMessage function processes incoming CDP traffic (lines 86–90), it identifies events that lack an id property (indicating they are notifications rather than responses) and are not consumed by active subscribers. These events are pushed onto the events array immediately after processing.

Following each insertion, the code evaluates events.length > MAX_BUFFERED_EVENTS. If the buffer exceeds capacity, the oldest entries are removed via events.splice(0, events.length - MAX_BUFFERED_EVENTS) (lines 88–90). This creates a sliding-window FIFO buffer where fresh events displace stale ones, maintaining a rolling history of the most recent 10,000 CDP notifications.

Atomic Drain Operation

Consumers retrieve accumulated events through drainBrowserEvents() defined at lines 64–66. This function returns a shallow copy of the current buffer and clears the original events array in a single operation, ensuring that no events are lost between retrieval and clearing. The atomic nature of this operation prevents race conditions when multiple consumers or teardown routines attempt to flush the queue simultaneously.

Consumer Interaction Patterns

The queue serves two distinct consumption models: immediate reactive handlers and deferred batch processing.

Real-Time Event Subscriptions

The subscribeBrowserEvent API allows functions to receive matching CDP events immediately upon arrival. According to the guard logic at line 86, these events are still stored in the buffer after delivery unless they are screencast frames (which are filtered to avoid flooding the queue with image data). This dual behavior ensures that real-time consumers can act on events while the buffer retains data for later inspection or testing.

import { subscribeBrowserEvent } from 'ego-browser';

const unsub = subscribeBrowserEvent('Network.responseReceived', undefined, (e) => {
  console.log('Response:', e.params?.response?.url);
});

// later, to stop listening
unsub();

Predicate-Based Waiting

For scenarios requiring synchronization on specific CDP states, waitForBrowserEvent registers predicates that resolve when a buffered event satisfies the condition (lines 73–106). When a match occurs, the waiter is resolved and removed from the registry, allowing agents to block until specific navigation or network events occur without polling.

import { waitForBrowserEvent } from 'ego-browser';

const navigation = await waitForBrowserEvent(
  (ev) => ev.method === 'Page.loadEventFired',
  5000,
);
console.log('Page loaded at', navigation.timestamp);

Runtime Integration and Auxiliary Files

The buffered event queue integrates with the broader runtime through auxiliary modules. The format.ts file defines CLI commands that expose the "Drain buffered page/CDP events" functionality to the runtime interface, while output-sink.ts demonstrates how the buffered output—including any remaining events—is flushed during process teardown to prevent data loss on exit.

import { drainBrowserEvents } from 'ego-browser';

const allEvents = drainBrowserEvents(); // empties the queue
console.log('Buffered CDP events count:', allEvents.length);

Summary

• Memory-bound buffering: The MAX_BUFFERED_EVENTS constant enforces a strict 10,000 event limit, protecting long-running agents from memory exhaustion.
• FIFO sliding window: Excess events are truncated from the front of the array using splice(), ensuring the buffer always contains the most recent CDP history.
• Atomic retrieval: drainBrowserEvents() copies and clears the queue in one operation, supporting race-free batch processing.
• Dual consumption: Events flow to immediate subscribers while remaining available for later draining, with special handling to exclude screencast frames from storage.

Frequently Asked Questions

What happens when the buffered event queue exceeds 10,000 events?

When the events array length surpasses MAX_BUFFERED_EVENTS (10,000), the runtime executes events.splice(0, events.length - MAX_BUFFERED_EVENTS) at lines 88–90 of browser-runtime.ts. This removes the oldest entries from the front of the array, maintaining a sliding window of the most recent 10,000 CDP events.

How do I retrieve all buffered events without losing data?

Call drainBrowserEvents() from the public API. According to lines 64–66 in browser-runtime.ts, this function returns a copy of the current buffer and clears the original events array atomically, ensuring you capture every event without race conditions.

Are subscribed events still stored in the buffer?

Yes. When you register a subscriber via subscribeBrowserEvent, the matching event is delivered to your callback and also pushed to the buffer (line 86), with the exception of screencast frames which are filtered out to prevent image data from consuming the 10,000 slot allocation.

Where is the capacity limit defined in the source code?

The 10,000 event capacity is defined by the MAX_BUFFERED_EVENTS constant at lines 6–9 of browser-runtime.ts in the citrolabs/ego-lite repository. This value is checked after every event insertion in the handleMessage function to enforce the memory ceiling.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}