# How gstack Uses Circular Buffers for High-Performance Logging Architecture

> Discover how gstack leverages circular buffers for a high-performance logging architecture. Achieve O(1) write speed and controlled memory with this innovative approach.

- Repository: [Garry Tan/gstack](https://github.com/garrytan/gstack)
- Tags: architecture
- Published: 2026-05-15

---

**gstack implements an in-memory ring buffer system using three fixed-size circular buffers (`consoleBuffer`, `networkBuffer`, and `dialogBuffer`) that asynchronously flush deltas to disk every second, ensuring O(1) write performance and bounded memory usage during long-running browser automation sessions.**

The gstack logging architecture centers on a lightweight, reusable **circular buffer** implementation designed to handle high-throughput events from headless Chromium processes without unbounded memory growth. By leveraging fixed-capacity ring buffers defined in [`browse/src/buffers.ts`](https://github.com/garrytan/gstack/blob/main/browse/src/buffers.ts), the system silently overwrites stale entries while preserving recent activity for debugging and telemetry.

## Core Data Structure: The CircularBuffer<T> Class

The foundation of gstack’s logging system lives in **[[`browse/src/buffers.ts`](https://github.com/garrytan/gstack/blob/main/browse/src/buffers.ts)](https://github.com/garrytan/gstack/blob/main/browse/src/buffers.ts)**, which exports a generic `CircularBuffer<T>` class optimized for constant-time operations.

The buffer maintains an internal **`head`** pointer and **`size`** counter, writing new entries at index `(head + size) % capacity`. When the buffer reaches capacity, the `head` advances and the oldest entry is overwritten, guaranteeing **O(1)** insertion regardless of buffer state.

Key methods include:

- **`push(entry: T)`** – Inserts a new entry, updates the monotonic `totalAdded` counter, and handles wrap-around automatically.
- **`toArray()`** – Returns all stored entries in **oldest-first** order for bulk dumps.
- **`last(n)`** – Retrieves the *n* most recent entries while maintaining chronological order.
- **`clear()`** – Resets `head` and `size` but preserves `totalAdded`, allowing the flush routine to maintain its cursor position when starting a fresh logging window.
- **`get(index)`** / **`set(index, entry)`** – Provide random access by logical index, primarily used by network-response matching logic.

The class exposes two critical public properties: **`length`** (current entry count) and **`totalAdded`** (running count of all pushes since initialization). The flush mechanism uses the delta between `totalAdded` and the last flushed count to determine what requires persistence.

## The Three gstack Log Streams

gstack instantiates three concrete buffer instances, each dedicated to a specific event class:

```typescript
const HIGH_WATER_MARK = 50_000;

export const consoleBuffer = new CircularBuffer<LogEntry>(HIGH_WATER_MARK);
export const networkBuffer = new CircularBuffer<NetworkEntry>(HIGH_WATER_MARK);
export const dialogBuffer = new CircularBuffer<DialogEntry>(HIGH_WATER_MARK);

```

Each buffer is capped at **50,000 entries**, a `HIGH_WATER_MARK` chosen to balance memory constraints against the need to capture large bursts of activity before overwrite occurs. The entry types—`LogEntry`, `NetworkEntry`, and `DialogEntry`—capture timestamps and domain-specific payloads such as log levels, HTTP methods, status codes, and dialog interaction types.

## Asynchronous Flush Routine to Disk

Durability is handled by **`flushBuffers()`** in **[[`browse/src/server.ts`](https://github.com/garrytan/gstack/blob/main/browse/src/server.ts)](https://github.com/garrytan/gstack/blob/main/browse/src/server.ts)**, which executes periodically (once per second) without blocking the main request path.

The routine performs the following steps:

1. **Calculates delta** for each buffer using `buffer.totalAdded - lastFlushed` to identify newly added entries.
2. **Retrieves fresh data** via `buffer.last(count)`, obtaining only the entries written since the previous flush.
3. **Formats entries** into single-line strings containing ISO timestamps, severity levels, and text payloads.
4. **Appends to disk** by writing the formatted block to corresponding files (`consoleLog`, `networkLog`, `dialogLog`) within the project’s `.gstack` directory.
5. **Updates cursors** by setting `last*Flushed` to the current `totalAdded` value, ensuring the next iteration processes only subsequent deltas.

A **`flushInProgress`** flag prevents overlapping flush operations, eliminating race conditions and file corruption risks during high-volume logging periods. This design achieves **eventual durability** while keeping the main automation thread responsive.

## Activity Tracking with Secondary Buffers

Beyond the three primary streams, gstack maintains an **activity buffer** in **[[`browse/src/activity.ts`](https://github.com/garrytan/gstack/blob/main/browse/src/activity.ts)](https://github.com/garrytan/gstack/blob/main/browse/src/activity.ts)**. This separate `CircularBuffer` instance records high-level command lifecycle events (start and end timestamps) for telemetry dashboards and UI feedback mechanisms, demonstrating the modularity of the ring-buffer approach across different subsystems.

## Why Circular Buffers? Architecture Benefits

gstack chose a ring-buffer architecture for four critical performance characteristics:

- **Constant-time insertion** – The daemon can push thousands of log entries per second without CPU overhead scaling with buffer size.
- **Bounded memory guarantees** – Fixed capacity prevents the memory leaks typical of unbounded arrays in long-running headless browser processes.
- **Efficient persistence** – By tracking `totalAdded` deltas, the system writes only new data rather than re-serializing entire buffers, minimizing I/O operations.
- **Deterministic retention** – Oldest data is discarded first, aligning with standard log retention policies that prioritize recent activity over historical entries.

## Working with gstack Buffers: Practical Examples

The following demonstrates how to interact with the logging system programmatically:

```typescript
import {
  consoleBuffer,
  networkBuffer,
  dialogBuffer,
  addConsoleEntry,
  addNetworkEntry,
  addDialogEntry,
  LogEntry,
  NetworkEntry,
  DialogEntry,
} from './buffers';

// Add a console log entry
addConsoleEntry({
  timestamp: Date.now(),
  level: 'info',
  text: 'Page loaded successfully',
});

// Record a network request
addNetworkEntry({
  timestamp: Date.now(),
  method: 'GET',
  url: 'https://example.com/api/data',
  status: 200,
  duration: 123,
  size: 4567,
});

// Log a dialog interaction
addDialogEntry({
  timestamp: Date.now(),
  type: 'alert',
  message: 'Welcome!',
  action: 'accepted',
});

// Retrieve the 10 most recent console entries (oldest-first)
const recentConsole = consoleBuffer.last(10);
console.log('Recent console entries:', recentConsole);

// Manually trigger flush (normally handled by periodic timer)
await flushBuffers(); // Exported from server.ts in the daemon

```

## Summary

- gstack’s logging relies on a reusable **`CircularBuffer<T>`** class in [`browse/src/buffers.ts`](https://github.com/garrytan/gstack/blob/main/browse/src/buffers.ts) that provides O(1) insertion and bounded memory usage.
- Three primary buffers—**`consoleBuffer`**, **`networkBuffer`**, and **`dialogBuffer`**—each store 50,000 entries of their respective event types.
- The **`flushBuffers()`** routine in [`browse/src/server.ts`](https://github.com/garrytan/gstack/blob/main/browse/src/server.ts) asynchronously persists delta updates to disk every second using the `totalAdded` counter to track new entries.
- An additional **`activityBuffer`** in [`browse/src/activity.ts`](https://github.com/garrytan/gstack/blob/main/browse/src/activity.ts) handles high-level telemetry using the same circular buffer implementation.
- The architecture prevents memory leaks in long-running Chromium processes while ensuring recent log data remains available for debugging.

## Frequently Asked Questions

### What is the maximum capacity of gstack log buffers?

Each buffer is initialized with a `HIGH_WATER_MARK` of **50,000 entries**. When this limit is reached, new entries overwrite the oldest data in O(1) time, ensuring memory usage never exceeds the fixed allocation regardless of how long the daemon runs.

### How does gstack prevent log data loss during high traffic?

The **`flushInProgress`** flag serializes flush operations to prevent file corruption, while the **`totalAdded`** counter ensures no entries are skipped during persistence. Even if the flush interval overlaps with high-volume writes, the delta calculation captures all entries added since the last successful write.

### Where are gstack log files stored on disk?

Flushed entries append to files within the project's **`.gstack`** directory. The specific files are `consoleLog` for browser console output, `networkLog` for HTTP request/response data, and `dialogLog` for JavaScript dialog interactions.

### Can I configure the circular buffer size in gstack?

The buffer capacity is hardcoded as `HIGH_WATER_MARK = 50_000` in [`browse/src/buffers.ts`](https://github.com/garrytan/gstack/blob/main/browse/src/buffers.ts). To modify this value, you must edit the source constant and rebuild the project, as the fixed size is fundamental to the memory-bounding guarantees of the logging architecture.