How the Buffered Event System Handles Overflow in Long-Running Tasks in Ego-Browser

Ego-Browser caps its CDP event buffer at 10,000 entries and automatically discards oldest events using splice() when the limit is exceeded, ensuring memory stays bounded during hours-long automation sessions.

The buffered event system in Ego-Browser manages a live stream of Chrome DevTools Protocol (CDP) events that automation scripts can query, filter, and wait on. For long-running tasks—such as multi-step scraping workflows or persistent browser sessions—this system must prevent unbounded memory growth. Here's how the implementation in citrolabs/ego-lite achieves this.

---

Buffer Configuration and Hard Limits

The overflow protection starts with explicit constants. In src/browser-runtime.ts, the module declares a fixed-size structure:

const MAX_BUFFERED_EVENTS = 10000;
const events: any[] = [];

This 10,000 event hard cap is defined at lines 6–9 of package/ego-browser/src/browser-runtime.ts. The array acts as a circular buffer in practice: new events flow in constantly, but the length deliberately never exceeds this threshold.

---

Event Ingestion and Automatic Truncation

Every CDP message flows through handleMessage, which processes the raw WebSocket payload from Chrome. After optional delivery to subscribed listeners, the event is pushed onto the buffer:

events.push(data);
if (events.length > MAX_BUFFERED_EVENTS) {
  events.splice(0, events.length - MAX_BUFFERED_EVENTS);
}

This three-line sequence at lines 86–90 in src/browser-runtime.ts is the core overflow mechanism. The splice(0, events.length - MAX_BUFFERED_EVENTS) call removes the oldest entries from the front of the array whenever the cap is exceeded. Because this check runs on every single event, the buffer cannot grow beyond 10,001 entries at any moment—even under extreme load from rapid CDP traffic.

---

Screencast Frame Exclusion to Reduce Memory Pressure

Large binary payloads receive special handling. When an event has already been delivered to a subscriber and its method is Page.screencastFrame, the code skips the events.push() entirely. This prevents duplicate storage of heavy screenshot or video frame data, which would otherwise consume disproportionate memory relative to typical CDP events.

The logic at lines 86–87 implements this optimization:

// Pseudocode of the actual branching
if (subscribersNotified && data.method === 'Page.screencastFrame') {
  // Skip buffering—already delivered, don't duplicate large frame
} else {
  events.push(data);
}

---

Draining and Resetting the Buffer

Helper functions expose controlled access to accumulated events. The drainBrowserEvents() function returns the current snapshot and clears the array:

export function drainBrowserEvents(): any[] {
  const snapshot = [...events];
  events.length = 0;  // Reset in place
  return snapshot;
}

This implementation at lines 64–66 allows automation scripts to periodically harvest events for analysis or logging, then start fresh with zero buffered history. Frequent draining further reduces the practical memory footprint, since splice() only triggers when the 10,000 limit is actively hit.

---

Using the Buffer in Practice

Here is a complete pattern for working with the bounded event system in long-running scripts:

import { 
  subscribeBrowserEvent, 
  waitForBrowserEvent, 
  drainBrowserEvents 
} from 'ego-browser';

// Continuous listener for console messages
const unsubscribe = subscribeBrowserEvent(
  'Runtime.consoleAPICalled',
  undefined,
  (evt) => console.log('Console:', evt.params)
);

// Periodic harvesting in a long-running loop
setInterval(() => {
  const recent = drainBrowserEvents();
  if (recent.length > 0) {
    console.log(`Drained ${recent.length} events, buffer reset`);
    // Process or persist `recent` as needed
  }
}, 30000); // Every 30 seconds

// Targeted wait with timeout for specific navigation
await waitForBrowserEvent(
  (e) => e.method === 'Page.loadEventFired',
  15000
);

---

Key Source Files

File	Responsibility
package/ego-browser/src/browser-runtime.ts	Core CDP transport, MAX_BUFFERED_EVENTS constant, handleMessage, and splice-based truncation
package/ego-browser/src/state.ts	Runtime state including sessionId and defaultTimeout used during buffer operations
package/ego-browser/src/helpers.ts (via helperContext)	High-level waitForBrowserEvent and drainBrowserEvents consumed by automation scripts

---

Summary

• Hard cap: MAX_BUFFERED_EVENTS = 10000 prevents unbounded array growth
• Automatic cleanup: splice(0, events.length - MAX_BUFFERED_EVENTS) discards oldest entries on every overflow
• Frame optimization: Page.screencastFrame events skip buffering when already delivered
• Manual relief: drainBrowserEvents() returns and clears the buffer for fresh collection
• Long-running safety: These mechanisms combine to keep memory stable across hours of CDP traffic

---

Frequently Asked Questions

What happens when the 10,000 event limit is reached?

The oldest events are immediately removed via splice(0, events.length - MAX_BUFFERED_EVENTS) to maintain exactly 10,000 buffered entries. This happens automatically in handleMessage at line 88–90 of src/browser-runtime.ts, with no data loss for subscribers already notified.

Can I configure the buffer size?

Currently, MAX_BUFFERED_EVENTS is a hardcoded constant at line 6 of src/browser-runtime.ts. The value is not exposed through the public API, so scripts must work within the 10,000 event limit or call drainBrowserEvents() more frequently.

Why are screencast frames treated differently?

Page.screencastFrame payloads contain large binary image data. Skipping their duplication in the buffer prevents memory bloat from video streams, while still allowing real-time delivery to active subscribers.

How do I retrieve buffered events programmatically?

Import drainBrowserEvents from ego-browser. This returns all accumulated CDP messages since the last drain and resets the internal array, effectively giving you a sliding window of recent browser activity with controlled memory usage.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}