# Memory Usage of the Async Logging Queue in spdlog: Implementation and Optimization

> Understand spdlog's async logging queue memory usage. Learn how the lock-free MPMC queue stores messages and calculate its footprint. Optimize your logging performance now.

- Repository: [Gabi Melman/spdlog](https://github.com/gabime/spdlog)
- Tags: performance
- Published: 2026-07-25

---

**spdlog's asynchronous logging queue stores full copies of formatted log messages in a lock-free MPMC blocking queue, consuming approximately `queue_capacity × (average_message_size + overhead)` bytes—roughly 8 MiB with default settings (8192 items) and 1 KB messages.**

The `gabime/spdlog` library implements high-performance asynchronous logging through a dedicated thread pool and lock-free queue architecture. Understanding the memory usage of the async logging queue is essential for capacity planning in high-throughput applications. This analysis examines the internal data structures, calculation methods, and configuration options that control memory consumption in the spdlog source code.

## Architecture of the Async Logging Queue

### Global Thread Pool Initialization

spdlog creates a global thread pool lazily when the first async logger is instantiated. According to the source code in [`include/spdlog/async.h`](https://github.com/gabime/spdlog/blob/main/include/spdlog/async.h) (lines 27-48), the `async_factory_impl` checks the registry and constructs a `details::thread_pool` with `details::default_async_q_size` (8192 items) and one worker thread by default.

### Queue Storage Structure

The thread pool maintains an instance `q_` of type `details::mpmc_blocking_queue<async_msg>`, defined in [`include/spdlog/details/thread_pool.h`](https://github.com/gabime/spdlog/blob/main/include/spdlog/details/thread_pool.h) (lines 72-76). This lock-free multi-producer-multi-consumer (MPMC) queue serves as the central message buffer between producer threads and the logging worker thread.

### Message Payload Storage

Each queue entry is an `async_msg` object (defined in [`include/spdlog/details/thread_pool.h`](https://github.com/gabime/spdlog/blob/main/include/spdlog/details/thread_pool.h), lines 25-68) containing:

- A `log_msg_buffer` instance (which internally holds a `memory_buf_t`)
- An `async_logger_ptr` (8 bytes on 64-bit systems)
- An `async_msg_type` enum

The `log_msg_buffer` class (found in [`include/spdlog/details/log_msg_buffer.h`](https://github.com/gabime/spdlog/blob/main/include/spdlog/details/log_msg_buffer.h), lines 11-18) creates a full copy of the log message payload. Unlike `log_msg` which only holds string views referencing stack data, `log_msg_buffer` allocates internal storage to ensure the message survives the asynchronous hand-off.

## Calculating Memory Consumption

The total memory footprint follows this formula:

```text
memory_used ≈ queue_capacity × (average_formatted_message_size + sizeof(async_msg))

```

With the default capacity of 8192 items:

- A workload generating 200-byte messages consumes approximately 200 × 8192 = ~1.6 MiB plus overhead
- A workload with 1 KB messages temporarily allocates ~8 MiB (8192 × 1 KB) plus overhead

The dominant factor is the formatted message buffer. The `async_msg` structure adds approximately 24 bytes of overhead per entry on 64-bit systems.

## Configuring Queue Memory Limits

### Adjust Queue Size

Call `spdlog::init_thread_pool(q_size, thread_count)` before creating any async loggers to override the default 8192-item capacity. Reducing `q_size` lowers the maximum memory footprint but increases the risk of blocking or message loss under heavy load.

```cpp
// Configure 4096 queue items with 2 worker threads
spdlog::init_thread_pool(4096, 2);
auto logger = spdlog::create_async<spdlog::sinks::stdout_sink_mt>("async_logger");

```

### Configure Overflow Policies

The `async_overflow_policy` determines behavior when the queue fills:

- **`block`** (default): Producer threads block until space is available
- **`overrun_oldest`**: Discard the oldest pending messages to make room

```cpp
// Non-blocking logger that drops oldest messages when queue is full
auto nb_logger = spdlog::create_async_nb<spdlog::sinks::basic_file_sink_mt>(
    "nb_logger", "log.txt");

```

### Optimize Message Size

Since spdlog does not truncate messages automatically, keeping formatted log lines concise reduces per-entry memory consumption proportionally.

## Runtime Monitoring

Inspect queue state through the global thread pool instance:

```cpp
size_t queued = spdlog::thread_pool()->queue_size();      // Current items in queue
size_t overrun = spdlog::thread_pool()->overrun_counter(); // Dropped due to overrun_oldest
size_t discard = spdlog::thread_pool()->discard_counter(); // Ignored due to other reasons

```

These metrics help identify memory pressure and tuning opportunities in production environments.

## Summary

- spdlog's async queue stores **full message copies** in `async_msg` objects within an `mpmc_blocking_queue`
- Default configuration allocates **8192 queue slots**, potentially consuming ~8 MiB with 1 KB messages
- Memory usage scales linearly: `capacity × (message_size + ~24 bytes overhead)`
- Control footprint via `spdlog::init_thread_pool()` to adjust capacity
- Choose between **blocking** (backpressure) and **overrun_oldest** (message loss) overflow policies
- Monitor runtime metrics using `thread_pool()->queue_size()` and counter methods

## Frequently Asked Questions

### How much memory does spdlog's async queue use by default?

With the default queue size of 8192 items, memory consumption depends on your average log message size. For 1 KB formatted messages, expect approximately 8 MiB of heap allocation, plus roughly 24 bytes of overhead per entry for the `async_msg` structure.

### Can I reduce the memory footprint of async logging?

Yes. Call `spdlog::init_thread_pool(q_size, thread_count)` with a smaller `q_size` value before creating async loggers. Alternatively, use shorter log messages, or switch to synchronous logging for low-volume loggers to eliminate queue allocation entirely.

### What happens when the async queue fills up?

By default, producer threads **block** until the worker thread consumes entries. If you create the logger using `create_async_nb` or specify `async_overflow_policy::overrun_oldest`, the oldest pending messages are silently discarded to make room for new entries.

### Does spdlog reuse memory buffers in the async queue?

No. Each `async_msg` entry allocates its own `memory_buf_t` storage to hold the formatted message copy. The queue pre-allocates slots for the `async_msg` objects themselves, but the message payload buffers are dynamically sized based on each log entry's content.