# How is Data Persisted in OpenRaft: A Complete Guide to Log and State Machine Storage

> Discover how OpenRaft persists data using RaftLogStorage and RaftStateMachine. Learn about durable flushing of votes, log entries, and snapshots for robust state management.

- Repository: [Databend Labs/openraft](https://github.com/databendlabs/openraft)
- Tags: deep-dive
- Published: 2026-02-28

---

**OpenRaft persists data through two distinct storage traits—`RaftLogStorage` for the distributed log and `RaftStateMachine` for application state—requiring implementations to durably flush votes, log entries, and snapshots before acknowledging completion.**

OpenRaft, the high-performance Raft consensus implementation maintained by `databendlabs/openraft`, delegates all durability concerns to user-provided storage backends. Understanding how data is persisted in OpenRaft requires examining the separation between log storage and state-machine storage, the specific durability contracts each trait demands, and the snapshot lifecycle that enables crash recovery.

## The Two-Layer Persistence Architecture

OpenRaft splits persistence responsibilities into two orthogonal components to maximize flexibility:

- **`RaftLogStorage`** ([`openraft/src/storage/v2/raft_log_storage.rs`](https://github.com/databendlabs/openraft/blob/main/openraft/src/storage/v2/raft_log_storage.rs)): Handles the **Raft log**—votes, log entries, and the optional committed index. This must survive crashes to prevent double-voting or log loss.
- **`RaftStateMachine`** ([`openraft/src/storage/v2/raft_state_machine.rs`](https://github.com/databendlabs/openraft/blob/main/openraft/src/storage/v2/raft_state_machine.rs)): Handles **application state** and **snapshots**. This can be either fully persistent (e.g., RocksDB) or transient with snapshot-based recovery.

This separation allows deployments to optimize log storage for write-heavy append workloads while using different backends (memory, RocksDB, custom databases) for the state machine.

## Persisting the Raft Log with RaftLogStorage

The `RaftLogStorage` trait defines three critical durability points that implementations must honor to guarantee safety.

### Saving the Vote

Before responding to any RPC, a node must persist its current term and voted-for candidate. The `save_vote` method must not return until the vote is flushed to durable storage:

```rust
async fn save_vote(&mut self, vote: &VoteOf<C>) -> Result<(), io::Error>;

```

Failure to durably save the vote can lead to split-brain scenarios after a crash, where a node might vote twice in the same term.

### Appending Log Entries

The `append` method receives new log entries and a completion callback. The contract requires that `callback.io_completed()` is invoked **only after** entries are durably written:

```rust
async fn append<I>(&mut self, entries: I, callback: IOFlushed<C>) -> Result<(), io::Error>
where I: IntoIterator<Item = C::Entry> + OptionalSend;

```

This callback mechanism allows OpenRaft to pipeline log appends while ensuring durability before committing entries to the state machine.

### Optional Commit Index Persistence

OpenRaft optionally allows storing the last committed log index via `save_committed`:

```rust
async fn save_committed(&mut self, _committed: Option<LogIdOf<C>>) -> Result<(), io::Error> {
    Ok(())
}

```

Persisting the committed index enables a node to resume with accurate `RaftMetrics.last_applied` after restart and prevents re-applying stale membership configurations. The `memstore` implementation demonstrates toggling this feature via an `AtomicBool`:

```rust
pub fn enable_save_committed(&self, enable: bool) {
    self.enable_saving_committed.store(enable, Ordering::Relaxed);
}

```

## State Machine Persistence and Snapshots

While the log storage requires immediate durability, the state machine offers more flexibility through the `RaftStateMachine` trait.

### Applied State Tracking

The `applied_state` method returns the last applied log ID and membership configuration, allowing OpenRaft to determine where to resume applying logs after restart:

```rust
async fn applied_state(&mut self) -> Result<(Option<LogIdOf<C>>, StoredMembership<C>), io::Error>;

```

### Applying Entries

The `apply` method receives a stream of committed entries. Implementations can choose their persistence strategy:

- **Immediate persistence**: Flush to disk during each `apply` call (slower but simpler recovery).
- **Deferred persistence**: Batch writes in memory, relying on snapshots for crash recovery (higher throughput).

### Snapshot Lifecycle

Snapshots provide full state-machine backups for recovery and log compaction. The lifecycle involves four key methods:

1. **`try_create_snapshot_builder`**: Initiates snapshot creation (e.g., copying RocksDB files).
2. **`begin_receiving_snapshot`**: Returns a `Cursor` for writing incoming snapshot data from the leader.
3. **`install_snapshot`**: Persists the received snapshot to disk and updates state.
4. **`get_current_snapshot`**: Returns the current snapshot for replication to lagging nodes.

The RocksDB example in [`examples/rocksstore/src/state_machine.rs`](https://github.com/databendlabs/openraft/blob/main/examples/rocksstore/src/state_machine.rs) demonstrates snapshot-only persistence, where the state machine writes to RocksDB but relies on periodic snapshots for crash recovery rather than WAL (Write-Ahead Logging) for every update.

## Key Source Files for OpenRaft Persistence

| File | Purpose |
|------|---------|
| [`openraft/src/storage/v2/raft_log_storage.rs`](https://github.com/databendlabs/openraft/blob/main/openraft/src/storage/v2/raft_log_storage.rs) | Defines `RaftLogStorage` trait with `save_vote`, `append`, and `save_committed` methods. |
| [`openraft/src/storage/v2/raft_state_machine.rs`](https://github.com/databendlabs/openraft/blob/main/openraft/src/storage/v2/raft_state_machine.rs) | Defines `RaftStateMachine` trait for applied state, entry application, and snapshot management. |
| [`stores/memstore/src/lib.rs`](https://github.com/databendlabs/openraft/blob/main/stores/memstore/src/lib.rs) | In-memory reference implementation showing optional `save_committed` toggling and callback-based `append`. |
| [`examples/rocksstore/src/state_machine.rs`](https://github.com/databendlabs/openraft/blob/main/examples/rocksstore/src/state_machine.rs) | Production-grade RocksDB implementation demonstrating snapshot-based persistence and state machine recovery. |
| [`openraft/src/docs/data/log_pointers.md`](https://github.com/databendlabs/openraft/blob/main/openraft/src/docs/data/log_pointers.md) | Documentation explaining the relationship between `flushed`, `committed`, and `applied` indices. |

## Summary

- **OpenRaft delegates all persistence to user-implemented traits**, separating log storage (`RaftLogStorage`) from state-machine storage (`RaftStateMachine`).
- **Log durability requires** synchronous flushing of votes before RPC returns and callback confirmation after `append` completes.
- **State machines can choose** between immediate persistence (every `apply` flushes) or snapshot-based recovery (periodic snapshots with transient state).
- **Optional `save_committed`** allows nodes to persist the last committed index, improving metric accuracy and preventing membership re-application after crashes.
- **Snapshots** provide the primary recovery mechanism for state machines, with `install_snapshot` called automatically on startup if a snapshot exists.

## Frequently Asked Questions

### What is the difference between RaftLogStorage and RaftStateMachine?

`RaftLogStorage` persists the **consensus log**—votes, log entries, and optionally the committed index—ensuring Raft safety properties survive crashes. `RaftStateMachine` persists **application data** and handles snapshots, allowing flexibility in how user data is stored (memory, RocksDB, etc.) while ensuring the state machine can recover to the last applied log index.

### Is the committed index required to be persisted?

No, persisting the committed index via `RaftLogStorage::save_committed` is **optional**. OpenRaft provides a default no-op implementation. However, persisting it improves recovery by allowing `RaftMetrics.last_applied` to survive restarts and prevents re-applying stale membership configurations. The `memstore` example shows how to toggle this feature at runtime.

### How does OpenRaft recover after a crash?

Recovery follows a specific sequence: First, OpenRaft restores the log from `RaftLogStorage`, recovering the vote and log entries up to the last `append`. If `save_committed` was implemented, it also knows the last committed index. Then, it calls `RaftStateMachine::install_snapshot` with any existing snapshot file to restore application state. Finally, it replays any log entries after the snapshot’s last applied index via `apply`, bringing the state machine to the current committed state.

### Can I use different storage backends for logs and state machine?

Yes, the trait-based architecture explicitly supports mixing storage backends. For example, you can store the Raft log in a high-performance write-ahead log (WAL) or dedicated disk while running the state machine in memory for speed (relying on snapshots for recovery), or vice versa. The `examples/rocksstore` demonstrates using RocksDB for both, while `stores/memstore` shows pure in-memory operation, illustrating the flexibility of the persistence model.