How tui-logger Handles Lost Messages When the Circular Buffer Overflows

tui-logger detects buffer overflows by comparing total event pushes against actual stored elements in src/logger/inner.rs, then injects a synthetic ExtLogRecord::overrun warning to report exactly how many messages were discarded.

The gin66/tui-logger crate provides a robust logging solution for terminal user interfaces using a dual-buffer architecture. When the internal circular buffer reaches capacity, the library explicitly reports message loss through structured warning entries rather than silently dropping data.

Two-Stage Buffer Architecture

tui-logger implements a two-stage buffering strategy that separates log ingestion from UI rendering:

• Hot buffer: A CircularBuffer<ExtLogRecord> defined in src/circular.rs and re-exported in src/lib.rs that receives every log entry via the raw_log method
• Main buffer: A secondary CircularBuffer<ExtLogRecord> that the terminal interface reads from

When the hot buffer fills to capacity, the move_events function transfers records to the main buffer. This design allows high-throughput logging without blocking the UI thread.

Detecting Buffer Overflows

The overflow detection logic resides in src/logger/inner.rs within the move_events implementation. During transfer, the code retrieves two key metrics from the hot buffer:

• total: received_events.total_elements() — The cumulative count of all push operations, including those that overwrote existing entries
• elements: received_events.len() — The count of distinct records currently stored in the buffer

When total > elements, the circular buffer has overwritten older records, indicating message loss.

if total > elements {
    // Too many events received, so some have been lost
    let new_log_entry =
        ExtLogRecord::overrun(reversed[reversed.len() - 1].timestamp, total, elements);
    reversed.push(new_log_entry);
}

Generating the Overrun Warning

When overflow is detected, tui-logger creates a synthetic warning record using the overrun helper defined in src/circular.rs. This method constructs an ExtLogRecord with level Warn and target "TuiLogger":

fn overrun(timestamp: DateTime<Local>, total: usize, elements: usize) -> Self {
    ExtLogRecord {
        timestamp,
        level: Level::Warn,
        target: "TuiLogger".to_string(),
        file: None,
        module_path: None,
        line: None,
        msg: format!(
            "There have been {} events lost, {} recorded out of {}",
            total - elements,
            elements,
            total
        ),
    }
}

The warning message explicitly states how many events were lost (total - elements), how many were preserved (elements), and the total number generated (total).

Behavior and Consequences

The overrun handling mechanism provides several important guarantees:

• Visible alerts: The warning appears in the TUI as a standard log line with level Warn, ensuring users immediately see data loss occurred
• Accurate statistics: The total_events counter increments by total, maintaining correct metrics even during overflow
• Graceful degradation: The logger never panics or crashes; it continues operating by discarding oldest messages while notifying the user

Configuration options in src/logger/api.rs allow tuning buffer depths via TuiLoggerBuilder methods like hot_depth() to minimize overflow frequency.

Practical Example

To observe the overflow behavior, configure a small hot buffer and generate excess log messages:

use tui_logger::{TuiLogger, TuiLoggerLevelOutput, TuiLoggerBuilder};

fn main() {
    // Initialize logger with minimal hot buffer to force overflow
    TuiLoggerBuilder::default()
        .hot_depth(3)               // Only retain 3 recent events in hot buffer
        .build()
        .unwrap();

    // Emit more messages than the buffer can hold
    for i in 0..10 {
        log::info!("message {}", i);
    }

    // Force transfer to trigger overflow detection
    TUI_LOGGER.move_events();
}

This produces output similar to:

WARN  TuiLogger  There have been 7 events lost, 3 recorded out of 10
INFO  mycrate    message 7
INFO  mycrate    message 8
INFO  mycrate    message 9

The warning clearly indicates that messages 0 through 6 were lost due to buffer constraints.

Summary

• tui-logger uses two circular buffers (hot and main) to manage log flow between producers and the UI
• Overflow detection occurs in src/logger/inner.rs by comparing total_elements() against len()
• The ExtLogRecord::overrun method in src/circular.rs generates synthetic warnings with precise loss metrics
• Lost message reports include the count lost, recorded, and total generated for complete transparency
• The system maintains accurate statistics and continues operating without panics during overflow conditions

Frequently Asked Questions

How does tui-logger detect that messages have been lost?

tui-logger detects lost messages in src/logger/inner.rs by comparing the return value of received_events.total_elements() (total pushes) against received_events.len() (current stored count). If the total exceeds the current length, the circular buffer has overwritten older entries, triggering warning generation.

Where is the overflow warning message formatted?

The warning message formatting resides in src/circular.rs within the ExtLogRecord::overrun method. This associated function constructs a complete log entry structure with level Warn, target "TuiLogger", and a detailed message string explaining exactly how many events were lost versus recorded.

Does tui-logger panic when the circular buffer overflows?

No. According to the implementation in src/logger/inner.rs, tui-logger handles overflows gracefully by creating a synthetic warning record and continuing operation. The system increments statistics correctly and preserves the most recent messages while discarding older ones without crashing.

How can I prevent message loss in tui-logger?

Configure larger buffer sizes using TuiLoggerBuilder::default().hot_depth(n) as defined in src/logger/api.rs. Increasing the hot buffer depth reduces the likelihood of overflow during high-throughput logging scenarios, though the overrun warning system ensures you are notified if capacity is exceeded regardless.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}