How gstack Uses Circular Buffers for High-Performance Logging Architecture

gstack implements an in-memory ring buffer system using three fixed-size circular buffers (consoleBuffer, networkBuffer, and dialogBuffer) that asynchronously flush deltas to disk every second, ensuring O(1) write performance and bounded memory usage during long-running browser automation sessions.

The gstack logging architecture centers on a lightweight, reusable circular buffer implementation designed to handle high-throughput events from headless Chromium processes without unbounded memory growth. By leveraging fixed-capacity ring buffers defined in browse/src/buffers.ts, the system silently overwrites stale entries while preserving recent activity for debugging and telemetry.

Core Data Structure: The CircularBuffer Class

The foundation of gstack’s logging system lives in [browse/src/buffers.ts](https://github.com/garrytan/gstack/blob/main/browse/src/buffers.ts), which exports a generic CircularBuffer<T> class optimized for constant-time operations.

The buffer maintains an internal head pointer and size counter, writing new entries at index (head + size) % capacity. When the buffer reaches capacity, the head advances and the oldest entry is overwritten, guaranteeing O(1) insertion regardless of buffer state.

Key methods include:

• push(entry: T) – Inserts a new entry, updates the monotonic totalAdded counter, and handles wrap-around automatically.
• toArray() – Returns all stored entries in oldest-first order for bulk dumps.
• last(n) – Retrieves the n most recent entries while maintaining chronological order.
• clear() – Resets head and size but preserves totalAdded, allowing the flush routine to maintain its cursor position when starting a fresh logging window.
• get(index) / set(index, entry) – Provide random access by logical index, primarily used by network-response matching logic.

The class exposes two critical public properties: length (current entry count) and totalAdded (running count of all pushes since initialization). The flush mechanism uses the delta between totalAdded and the last flushed count to determine what requires persistence.

The Three gstack Log Streams

gstack instantiates three concrete buffer instances, each dedicated to a specific event class:

const HIGH_WATER_MARK = 50_000;

export const consoleBuffer = new CircularBuffer<LogEntry>(HIGH_WATER_MARK);
export const networkBuffer = new CircularBuffer<NetworkEntry>(HIGH_WATER_MARK);
export const dialogBuffer = new CircularBuffer<DialogEntry>(HIGH_WATER_MARK);

Each buffer is capped at 50,000 entries, a HIGH_WATER_MARK chosen to balance memory constraints against the need to capture large bursts of activity before overwrite occurs. The entry types—LogEntry, NetworkEntry, and DialogEntry—capture timestamps and domain-specific payloads such as log levels, HTTP methods, status codes, and dialog interaction types.

Asynchronous Flush Routine to Disk

Durability is handled by flushBuffers() in [browse/src/server.ts](https://github.com/garrytan/gstack/blob/main/browse/src/server.ts), which executes periodically (once per second) without blocking the main request path.

The routine performs the following steps:

1. Calculates delta for each buffer using buffer.totalAdded - lastFlushed to identify newly added entries.
2. Retrieves fresh data via buffer.last(count), obtaining only the entries written since the previous flush.
3. Formats entries into single-line strings containing ISO timestamps, severity levels, and text payloads.
4. Appends to disk by writing the formatted block to corresponding files (consoleLog, networkLog, dialogLog) within the project’s .gstack directory.
5. Updates cursors by setting last*Flushed to the current totalAdded value, ensuring the next iteration processes only subsequent deltas.

A flushInProgress flag prevents overlapping flush operations, eliminating race conditions and file corruption risks during high-volume logging periods. This design achieves eventual durability while keeping the main automation thread responsive.

Activity Tracking with Secondary Buffers

Beyond the three primary streams, gstack maintains an activity buffer in [browse/src/activity.ts](https://github.com/garrytan/gstack/blob/main/browse/src/activity.ts). This separate CircularBuffer instance records high-level command lifecycle events (start and end timestamps) for telemetry dashboards and UI feedback mechanisms, demonstrating the modularity of the ring-buffer approach across different subsystems.

Why Circular Buffers? Architecture Benefits

gstack chose a ring-buffer architecture for four critical performance characteristics:

• Constant-time insertion – The daemon can push thousands of log entries per second without CPU overhead scaling with buffer size.
• Bounded memory guarantees – Fixed capacity prevents the memory leaks typical of unbounded arrays in long-running headless browser processes.
• Efficient persistence – By tracking totalAdded deltas, the system writes only new data rather than re-serializing entire buffers, minimizing I/O operations.
• Deterministic retention – Oldest data is discarded first, aligning with standard log retention policies that prioritize recent activity over historical entries.

Working with gstack Buffers: Practical Examples

The following demonstrates how to interact with the logging system programmatically:

import {
  consoleBuffer,
  networkBuffer,
  dialogBuffer,
  addConsoleEntry,
  addNetworkEntry,
  addDialogEntry,
  LogEntry,
  NetworkEntry,
  DialogEntry,
} from './buffers';

// Add a console log entry
addConsoleEntry({
  timestamp: Date.now(),
  level: 'info',
  text: 'Page loaded successfully',
});

// Record a network request
addNetworkEntry({
  timestamp: Date.now(),
  method: 'GET',
  url: 'https://example.com/api/data',
  status: 200,
  duration: 123,
  size: 4567,
});

// Log a dialog interaction
addDialogEntry({
  timestamp: Date.now(),
  type: 'alert',
  message: 'Welcome!',
  action: 'accepted',
});

// Retrieve the 10 most recent console entries (oldest-first)
const recentConsole = consoleBuffer.last(10);
console.log('Recent console entries:', recentConsole);

// Manually trigger flush (normally handled by periodic timer)
await flushBuffers(); // Exported from server.ts in the daemon

Summary

• gstack’s logging relies on a reusable CircularBuffer<T> class in browse/src/buffers.ts that provides O(1) insertion and bounded memory usage.
• Three primary buffers—consoleBuffer, networkBuffer, and dialogBuffer—each store 50,000 entries of their respective event types.
• The flushBuffers() routine in browse/src/server.ts asynchronously persists delta updates to disk every second using the totalAdded counter to track new entries.
• An additional activityBuffer in browse/src/activity.ts handles high-level telemetry using the same circular buffer implementation.
• The architecture prevents memory leaks in long-running Chromium processes while ensuring recent log data remains available for debugging.

Frequently Asked Questions

What is the maximum capacity of gstack log buffers?

Each buffer is initialized with a HIGH_WATER_MARK of 50,000 entries. When this limit is reached, new entries overwrite the oldest data in O(1) time, ensuring memory usage never exceeds the fixed allocation regardless of how long the daemon runs.

How does gstack prevent log data loss during high traffic?

The flushInProgress flag serializes flush operations to prevent file corruption, while the totalAdded counter ensures no entries are skipped during persistence. Even if the flush interval overlaps with high-volume writes, the delta calculation captures all entries added since the last successful write.

Where are gstack log files stored on disk?

Flushed entries append to files within the project's .gstack directory. The specific files are consoleLog for browser console output, networkLog for HTTP request/response data, and dialogLog for JavaScript dialog interactions.

Can I configure the circular buffer size in gstack?

The buffer capacity is hardcoded as HIGH_WATER_MARK = 50_000 in browse/src/buffers.ts. To modify this value, you must edit the source constant and rebuild the project, as the fixed size is fundamental to the memory-bounding guarantees of the logging architecture.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}