Event Buffering System Architecture in ego-browser: Understanding the 10K IOk Cap

The ego-browser event buffering system implements a FIFO queue in package/ego-browser/src/browser-runtime.ts that captures Chrome DevTools Protocol (CDP) events with a hard 10,000-item IOk cap, dropping excess events and emitting a warning when the buffer limit is reached to prevent memory overflow.

The citrolabs/ego-lite repository provides a deterministic browser automation environment where the ego-browser package mediates between asynchronous CDP streams and synchronous agent execution. Its event buffering architecture maintains an in-memory queue with strict memory safeguards, ensuring that high-volume browser events do not exhaust resources during intensive automation tasks.

Core Architecture Components

The FIFO Event Queue

At the heart of the system lies eventQueue, a simple JavaScript array acting as a first-in-first-out buffer. Located in package/ego-browser/src/browser-runtime.ts, this queue collects every CDP message emitted by the embedded ego-lite browser runtime. The queue decouples event production from consumption, allowing the browser to emit navigation, network, and DOM mutation events continuously while the agent script processes them at its own pace.

The 10K IOk Memory Cap

To prevent unbounded memory growth during high-frequency event bursts, the architecture enforces a hard limit of 10,000 buffered items (the IOk cap). Before each insertion, the runtime executes:

if (eventQueue.length >= 10_000) {
  logger.warn('IOk cap reached – dropping events');
  return;
}

When the queue reaches this threshold, subsequent events are silently discarded and a warning is logged. This deterministic safeguard ensures that even under extreme load—such as rapid page mutations or network storms—the process remains stable and avoids out-of-memory crashes.

Session Initialization and Event Listener

The ensureSession() function in browser-runtime.ts establishes the CDP session via ego.sendCDPMessage('Target.attachToTarget', …) and registers the primary event listener. Once attached, this listener operates asynchronously, pushing every incoming CDP message onto eventQueue independently of the main execution thread. This design ensures no events are lost during brief agent pauses or synchronous waits.

Event Flushing and Delivery

The internal flushEvents() method drains the queue, popping events and delivering them to awaiting consumers. This function executes after each user-script turn and during explicit synchronization points, moving events from the buffer to the agent's execution context where synchronous helpers can inspect them.

How the Buffering Flow Works

The event lifecycle follows a deterministic four-stage pipeline:

1. Capture: CDP events arrive asynchronously from the browser runtime via the listener established in ensureSession().
2. Validation: Before insertion, the runtime checks eventQueue.length against the 10,000-item IOk cap.
3. Storage: Valid events are pushed onto the FIFO queue; if the cap is exceeded, the event is discarded.
4. Consumption: The flushEvents() routine periodically empties the queue, delivering events to helper functions in src/helpers.ts that power synchronous automation primitives.

This flow guarantees that agents always operate on recent event data while maintaining predictable memory footprints bounded by the 10K limit.

Practical Implementation Examples

Consuming Buffered Events

Access the event stream using the waitForEvent helper, which polls flushed results from the queue:

import { waitForEvent } from 'ego-browser/helpers';

// Pause execution until a network response arrives
const response = await waitForEvent('Network.responseReceived', { 
  timeout: 5000 
});

The helper internally calls flushEvents(), retrying until the desired event appears or the timeout expires. If the IOk cap was reached earlier, older events may be missing from the buffer.

Monitoring Buffer State

For debugging, inspect the queue length directly (typically available in test environments):

// Log current buffer utilization
console.log(`Buffered events: ${eventQueue.length}`);
if (eventQueue.length > 9000) {
  console.warn('Approaching 10K IOk cap');
}

Adjusting the Buffer Limit

While the 10K default suits most automation workloads, you can modify the constant in browser-runtime.ts:

// In package/ego-browser/src/browser-runtime.ts
const EVENT_QUEUE_IOK_CAP = 20_000; // Increased from 10,000

After modification, rebuild the package with npm run build to apply the new limit. Note that higher values increase memory consumption proportionally to the size of buffered CDP messages.

Integration with Agent Subsystems

The buffering system serves multiple downstream modules:

• src/helpers.ts: Provides waitForEvent() and onMessage() wrappers that consume flushed events, abstracting raw CDP for agent scripts.
• src/driver/waits.ts: Implements deterministic waiting primitives relying on the buffered event stream to detect DOM stability and navigation completion.
• src/ref-map.ts / src/ref-state.ts: Snapshot and reference logic may enqueue navigation-finished events that are later consumed via the queue.
• src/driver/*: Pointer, keyboard, and navigation drivers trigger CDP commands that generate events; these are buffered for later consumption by agent helpers.

Summary

• Location: Core implementation resides in package/ego-browser/src/browser-runtime.ts with state references in src/state.ts.
• Mechanism: A FIFO eventQueue stores CDP events emitted by the ego-lite browser runtime.
• Safety: The IOk cap enforces a strict 10,000-item limit, logging warnings and dropping overflow to prevent memory exhaustion.
• Lifecycle: Events flow through capture (ensureSession()), validation (length check), storage (array push), and consumption (flushEvents()).
• Consumption: Synchronous helpers in src/helpers.ts drain the queue to enable blocking waits like waitForEvent().

Frequently Asked Questions

What happens when the event buffer reaches the 10,000-item IOk cap?

When eventQueue.length reaches 10,000, the runtime stops accepting new events. Incoming CDP messages are discarded immediately, and a warning is logged via the internal logger. Existing events remain in the buffer for consumption, but new arrivals are dropped until flushEvents() drains the queue below the threshold.

Where is the event buffering logic implemented in the ego-browser source code?

The primary implementation lives in package/ego-browser/src/browser-runtime.ts, specifically within the session management code that defines eventQueue and the ensureSession() function. The queue state is also referenced in src/state.ts, while consumption logic resides in src/helpers.ts and src/driver/waits.ts.

Can I increase the 10K event buffer limit for high-traffic automation?

Yes. The limit is defined as a constant in browser-runtime.ts (commonly referenced as the IOk value or 10_000). You can increase this numeric literal to exceed 10,000 items, then rebuild the package. Be aware that higher limits increase memory usage proportionally to the size and frequency of CDP messages.

How does ego-browser prevent event loss during script pauses?

The asynchronous event listener registered by ensureSession() continues receiving CDP messages into the eventQueue even when the agent script is paused at a breakpoint or awaiting a synchronous result. This decoupling ensures that browser events emitted during agent computation are retained (subject to the 10K cap) and become available when flushEvents() next executes.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}