# OpenRaft Storage Performance Tuning: A Complete Guide to Configuration and Optimization

> Optimize OpenRaft storage performance by tuning batch sizes, snapshot policies, and persistence settings. Learn how configuration impacts I/O and network throughput for efficient operation.

- Repository: [Databend Labs/openraft](https://github.com/databendlabs/openraft)
- Tags: performance
- Published: 2026-02-28

---

**OpenRaft storage performance tuning centers on adjusting batch sizes, snapshot policies, and persistence settings in the `Config` struct to match your disk I/O and network characteristics.**

OpenRaft is a high-performance Raft consensus implementation maintained by databendlabs/openraft that separates log storage from state machine storage through the `RaftLogStorage` and `RaftStateMachine` traits. By tuning parameters in [`openraft/src/config/config.rs`](https://github.com/databendlabs/openraft/blob/main/openraft/src/config/config.rs) and selecting appropriate store implementations, you can optimize throughput for SSD-backed production clusters or minimize latency for in-memory benchmarks.

## Understanding OpenRaft Storage Architecture

OpenRaft delegates all persistence to two pluggable interfaces: the **log store** (`RaftLogStorage`) for Raft log entries, and the **state machine store** (`RaftStateMachine`) for snapshots and applied state. The repository provides an in-memory implementation (`MemLogStore`/`MemStateMachine`) in [`stores/memstore/src/lib.rs`](https://github.com/databendlabs/openraft/blob/main/stores/memstore/src/lib.rs) for testing, and a RocksDB-backed implementation (`RocksLogStore`) in [`examples/rocksstore/src/log_store.rs`](https://github.com/databendlabs/openraft/blob/main/examples/rocksstore/src/log_store.rs) for production.

### Log Store Implementation Details

The `RaftLogStorage` trait requires methods like `append`, `purge`, and `get_log_state`. In [`stores/memstore/src/lib.rs`](https://github.com/databendlabs/openraft/blob/main/stores/memstore/src/lib.rs), `MemLogStore` stores entries in a `BTreeMap<u64, String>` with JSON serialization, invoking the `IOFlushed` callback immediately after insertion. Purging removes entries up to a given `LogId` and can be artificially delayed via `BlockOperation::PurgeLog`.

In [`examples/rocksstore/src/log_store.rs`](https://github.com/databendlabs/openraft/blob/main/examples/rocksstore/src/log_store.rs), `RocksLogStore` persists log entries in a RocksDB column family. Logs are stored with big-endian binary keys (`id_to_bin`) to preserve ordering. The `append` method writes each entry as a JSON payload, then spawns a thread that flushes the WAL and invokes the callback (`callback.io_completed(res)`). The `purge` method writes the *last-purged* log id as metadata (`meta::LastPurged`) and deletes the key range in one atomic range delete.

Both stores expose a `get_log_state` method that returns the `last_log_id` and the `last_purged_log_id`, which the Raft core uses to decide when to trigger snapshots or log compaction.

### State Machine and Snapshot Lifecycle

Snapshots are triggered by `Config::snapshot_policy` (e.g., `LogsSinceLast(N)`) when the committed index advances *N* entries beyond the last snapshot. The `RaftSnapshotBuilder` implementation in `MemStateMachine` serializes the entire state machine via `serde_json::to_vec`, reporting the payload size for observability. The `snapshot_max_chunk_size` parameter controls how this payload is fragmented for network transmission, while `max_in_snapshot_log_to_keep` determines how many logs persist after snapshot installation to balance recovery speed against disk usage.

## Key Configuration Parameters for Storage Performance

All tunables reside in [`openraft/src/config/config.rs`](https://github.com/databendlabs/openraft/blob/main/openraft/src/config/config.rs) within the `Config` struct. Adjusting these values allows you to optimize the trade-off between write latency, replication throughput, and snapshot frequency.

### Batching and Network Tuning

**`max_append_entries`** (`Option<usize>`, default 4096) sets the upper bound on entries written in a single I/O operation. Increasing this value to `Some(8192)` or higher reduces disk sync overhead on fast SSDs but increases latency for individual entries.

**`max_payload_entries`** (`usize`, default 300) controls how many entries a leader ships in a single `AppendEntries` RPC. Raising this to 500–1000 reduces round-trips on reliable, high-bandwidth networks.

### Snapshot Management

**`replication_lag_threshold`** (`u64`, default 10000) defines the log gap that triggers snapshot-based catch-up. Lowering this value forces snapshots earlier, which is beneficial when logs grow large, but increases snapshot build frequency.

**`snapshot_policy`** (`SnapshotPolicy`, default `LogsSinceLast(5000)`) determines when to trigger automatic snapshots. Setting this to `LogsSinceLast(2000)` for high-write workloads prevents unbounded log growth.

**`snapshot_max_chunk_size`** (`u64`, default 1MB) limits the byte size of snapshot chunks transmitted to followers. Increasing this to 8MB or 16MB reduces RPC count for large state machines.

### Log Retention and Purging

**`max_in_snapshot_log_to_keep`** (`u64`, default 1000) specifies how many logs to retain after a snapshot. Smaller values (e.g., 500) speed up purging and reduce disk usage, but may increase recovery time after a crash.

**`purge_batch_size`** (`u64`, default 256) sets the minimum number of applied logs removed in one batch. Larger batches (e.g., 512 or 1024) reduce the overhead of frequent purge operations.

### Persistence Optimization

**`enable_saving_committed`** (memstore only, default `true`) controls whether `MemLogStore` persists the committed index. Disabling this via `log_store.enable_saving_committed.store(false, std::sync::atomic::Ordering::Relaxed)` eliminates a write per commit, useful only for pure in-memory benchmarks where durability is not required.

## Production Tuning Examples

### Optimizing RocksDB for SSD Workloads

For SSD-backed deployments using `RocksLogStore`, combine OpenRaft configuration with aggressive batching:

```rust
use openraft::{Config, SnapshotPolicy};
use std::sync::Arc;

let mut cfg = Config::default();
cfg.max_append_entries = Some(8192);          // Large batches for SSD
cfg.max_payload_entries = 1_000;             // Reduce network round-trips
cfg.replication_lag_threshold = 5_000;        // Early snapshot trigger
cfg.snapshot_policy = SnapshotPolicy::LogsSinceLast(2_000);
cfg.snapshot_max_chunk_size = 8 * 1024 * 1024; // 8 MiB chunks
cfg.max_in_snapshot_log_to_keep = 500;
cfg.purge_batch_size = 200;

let cfg = Arc::new(cfg.validate()?);
// Initialize RocksDB log_store...

```

This configuration minimizes disk sync frequency while ensuring snapshots prevent unbounded log growth.

### High-Performance In-Memory Benchmarking

When measuring raw Raft overhead without persistence costs:

```rust
use openraft::{Config, SnapshotPolicy};
use std::sync::Arc;

let mut cfg = Config::default();
cfg.max_append_entries = Some(4_096);
cfg.max_payload_entries = 300;
cfg.snapshot_policy = SnapshotPolicy::Never; // Prevent automatic snapshots
let cfg = Arc::new(cfg.validate()?);

let (log_store, state_machine) = openraft::stores::memstore::new_mem_store();
// Disable committed index persistence for pure speed
log_store.enable_saving_committed.store(false, std::sync::atomic::Ordering::Relaxed);

```

Disabling `enable_saving_committed` removes the final write barrier per commit, yielding lower latency in transient benchmarks.

### Simulating Slow Storage with BlockConfig

The test harness provides `BlockConfig` to inject artificial latency:

```rust
use openraft::stores::memstore::{BlockConfig, BlockOperation};
use std::time::Duration;

let block = BlockConfig::default();
block.set_blocking(BlockOperation::BuildSnapshot, Duration::from_millis(200));
block.set_blocking(BlockOperation::PurgeLog, Duration::from_millis(50));

let (log_store, state_machine) = 
    openraft::stores::memstore::new_mem_store_with_block(block);

```

This simulates slow disks to validate Raft behavior under I/O pressure without hardware changes.

## Critical Source Files and Implementation Details

| Path | Purpose | Key Components |
|------|---------|----------------|
| [`openraft/src/config/config.rs`](https://github.com/databendlabs/openraft/blob/main/openraft/src/config/config.rs) | Runtime configuration | `Config` struct, `max_append_entries`, `snapshot_policy` |
| [`stores/memstore/src/lib.rs`](https://github.com/databendlabs/openraft/blob/main/stores/memstore/src/lib.rs) | In-memory storage reference | `MemLogStore`, `MemStateMachine`, `BlockConfig` |
| [`examples/rocksstore/src/log_store.rs`](https://github.com/databendlabs/openraft/blob/main/examples/rocksstore/src/log_store.rs) | Production RocksDB backend | `RocksLogStore`, WAL flushing, range deletes |
| [`benchmarks/minimal/src/store.rs`](https://github.com/databendlabs/openraft/blob/main/benchmarks/minimal/src/store.rs) | Benchmark harness store | Minimal in-memory implementation for throughput tests |
| [`tests/tests/replication/t62_follower_clear_restart_recover.rs`](https://github.com/databendlabs/openraft/blob/main/tests/tests/replication/t62_follower_clear_restart_recover.rs) | Recovery testing | Demonstrates `enable_saving_committed` behavior |
| [`tests/tests/state_machine/t10_total_order_apply.rs`](https://github.com/databendlabs/openraft/blob/main/tests/tests/state_machine/t10_total_order_apply.rs) | Ordering guarantees | Validates state machine apply order under load |

## Best Practices for Production Deployments

- **Start with conservative defaults** – The default `max_append_entries = 4096` and `max_payload_entries = 300` suit most SSD and Ethernet environments.

- **Measure commit latency first** – If latency spikes occur under load, reduce `max_append_entries` to avoid long write stalls rather than increasing batch sizes further.

- **Balance snapshot frequency** – Set `snapshot_policy = LogsSinceLast(N)` where *N* reflects your write rate; pair with `replication_lag_threshold` to prevent excessive snapshot builds for minor follower lag.

- **Minimize log retention** – Use `max_in_snapshot_log_to_keep = 500` (or lower) to reduce disk usage, accepting slightly longer recovery times after crashes.

- **Disable persistence only for benchmarks** – Keep `enable_saving_committed = true` in production; disable it only in `MemLogStore` for transient performance testing.

- **Tune RocksDB independently** – OpenRaft issues simple put/delete operations; optimize RocksDB's write buffer, compression, and compaction settings to match your hardware for dominant performance gains.

- **Monitor with tracing** – Enable `tracing` subscribers to observe `log compaction complete` events and snapshot build times, adjusting configuration iteratively based on real-time metrics.

## Summary

OpenRaft storage performance tuning revolves around configuring batch sizes, snapshot policies, and persistence settings in [`openraft/src/config/config.rs`](https://github.com/databendlabs/openraft/blob/main/openraft/src/config/config.rs) to align with your hardware capabilities. Key adjustments include increasing `max_append_entries` for SSD throughput, tuning `snapshot_policy` and `replication_lag_threshold` to balance log growth against snapshot overhead, and managing retention via `max_in_snapshot_log_to_keep` and `purge_batch_size`. For production RocksDB deployments, these parameters work alongside database-level optimization, while in-memory benchmarks can disable `enable_saving_committed` to eliminate persistence overhead.

## Frequently Asked Questions

### What is the optimal max_append_entries value for SSD storage?

For SSD-backed storage, setting `max_append_entries` to `Some(8192)` or higher in [`openraft/src/config/config.rs`](https://github.com/databendlabs/openraft/blob/main/openraft/src/config/config.rs) maximizes throughput by amortizing disk sync costs across many entries. However, if you observe increased commit latency or write stalls, reduce this value to `Some(4096)` or `Some(2048)` to prevent long-running I/O operations from blocking the Raft core. The optimal value depends on your specific SSD's IOPS capabilities and the latency requirements of your application.

### How does snapshot_policy affect cluster performance?

The `snapshot_policy` parameter, typically configured as `SnapshotPolicy::LogsSinceLast(N)` in `Config::snapshot_policy`, controls how frequently the state machine is serialized to disk. Aggressive policies with low *N* values prevent unbounded log growth and reduce recovery time for lagging followers, but consume CPU and I/O resources for frequent snapshot builds. Conservative policies reduce snapshot overhead but may cause excessive network traffic when synchronizing followers that have fallen far behind, requiring careful tuning alongside `replication_lag_threshold`.

### When should I disable enable_saving_committed?

You should only disable `enable_saving_committed` when using `MemLogStore` for pure in-memory benchmarking where durability is not required and you need to measure minimal commit latency. This is accomplished by calling `log_store.enable_saving_committed.store(false, std::sync::atomic::Ordering::Relaxed)` as shown in [`stores/memstore/src/lib.rs`](https://github.com/databendlabs/openraft/blob/main/stores/memstore/src/lib.rs). Never disable this in production deployments or when using persistent storage like RocksDB, as it risks losing committed log positions during process restarts and violates Raft safety guarantees.

### How do I tune OpenRaft for high-throughput benchmarking?

For high-throughput benchmarking, configure aggressive batching and disable automatic snapshots to measure raw replication overhead. Set `max_append_entries` to `Some(4096)` and `max_payload_entries` to `300` or higher, and use `SnapshotPolicy::Never` to prevent automatic snapshot interruptions. Initialize `MemLogStore` and disable `enable_saving_committed` to eliminate persistence overhead. Additionally, ensure `enable_heartbeat` and `enable_elect` are configured appropriately for your test topology (often disabled for single-node benchmarks) to isolate the storage layer performance.