# Storage Backend Requirements for OpenRaft: Implementing RaftLogStorage and RaftStateMachine Traits

> Explore OpenRaft storage backend requirements. Implement RaftLogStorage and RaftStateMachine traits for reliable log persistence and state machine snapshots with serialized writes and hole-free logs.

- Repository: [Databend Labs/openraft](https://github.com/databendlabs/openraft)
- Tags: deep-dive
- Published: 2026-02-28

---

**OpenRaft requires storage backends to implement two async traits—`RaftLogStorage` for log persistence and `RaftStateMachine` for state machine snapshots—guaranteeing serialized writes, hole-free logs, and immediate durability.**

OpenRaft, the high-performance Raft consensus library from `databendlabs/openraft`, strictly separates the consensus algorithm from concrete persistence mechanisms. To integrate a custom storage layer, developers must satisfy rigorous **storage backend requirements for OpenRaft** by implementing specific async traits that ensure correctness, durability, and thread safety across the distributed system.

## Core Storage Traits in OpenRaft

The storage layer in `openraft/src/storage/v2/` defines the contract between the Raft core and persistence. Two traits form the foundation of any backend implementation.

### RaftLogStorage Trait Requirements

Defined in [`openraft/src/storage/v2/raft_log_storage.rs`](https://github.com/databendlabs/openraft/blob/main/openraft/src/storage/v2/raft_log_storage.rs), **`RaftLogStorage<C>`** manages the replicated log, vote persistence, and committed log tracking. Implementations must provide several async methods with strict behavioral guarantees:

- **`get_log_state()`**: Returns `LogState` containing `last_purged_log_id` and `last_log_id` to establish the current log bounds.
- **`save_vote()`**: Persists the current vote to stable storage before returning, ensuring leader election safety.
- **`append()`**: Adds entries to the log, makes them immediately readable, and invokes the `IOFlushed` callback only after entries are durable on disk.
- **`truncate_after()`**: Removes entries after a specific log ID while maintaining the hole-free invariant.
- **`purge()`**: Removes entries up to a specific log ID, typically invoked after snapshot installation to reclaim space.

### RaftStateMachine Trait Requirements

Defined in [`openraft/src/storage/v2/raft_state_machine.rs`](https://github.com/databendlabs/openraft/blob/main/openraft/src/storage/v2/raft_state_machine.rs), **`RaftStateMachine<C>`** handles business logic application and snapshot management. Critical methods include:

- **`applied_state()`**: Must return the **last applied log ID** and **last applied membership**, which the Raft core uses for correctness checks during restarts.
- **`apply()`**: Applies log entries to the state machine, persists the last-applied log ID, executes business logic, and optionally triggers state machine persistence.
- **`begin_receiving_snapshot()`**, **`install_snapshot()`**, **`get_current_snapshot()`**: Manage snapshot reception, installation, and retrieval during replication and node restart.
- **`build_snapshot()`**: Creates a consistent point-in-time snapshot of the state machine.

## Critical Durability and Consistency Constraints

OpenRaft imposes strict architectural constraints documented directly in the trait definitions. Violating these invariants compromises consensus safety and leader election guarantees.

| Requirement | Implementation Detail | Source Location |
|---|---|---|
| **Serialized Writes** | All write operations (append, truncate, purge, vote) must be serialized so later writes never overtake earlier ones. | `RaftLogStorage` serialization comment in [`raft_log_storage.rs`](https://github.com/databendlabs/openraft/blob/main/raft_log_storage.rs) |
| **Hole-Free Logs** | The log must contain no gaps; `truncate_after` and `purge` must maintain contiguous indices. | `RaftLogStorage` "no hole" comment |
| **Vote Durability** | `save_vote` must persist the vote to stable storage before returning. | `save_vote` documentation |
| **General Durability** | Once a write returns, data must be durable (e.g., via `fsync` or database WAL). | `RaftLogStorage` durability paragraph |
| **Append Callback** | `append` must invoke the `IOFlushed` callback only after entries are persisted, while making entries readable immediately. | `append` documentation |
| **Committed Log Tracking** | Optional `save_committed`/`read_committed` for storing last committed log ID; required if snapshots are non-atomic. | `save_committed` docs in [`raft_log_storage.rs`](https://github.com/databendlabs/openraft/blob/main/raft_log_storage.rs) |
| **Thread Safety** | Implementations must be `Send + Sync` and async-compatible (`OptionalSend + OptionalSync`). | Trait bounds in both storage files |

## Production Storage Backend Options

The OpenRaft repository provides reference implementations demonstrating different durability levels and architectural patterns.

### In-Memory Storage (MemStore)

Located in [`examples/raft-kv-memstore/src/store/mod.rs`](https://github.com/databendlabs/openraft/blob/main/examples/raft-kv-memstore/src/store/mod.rs), the in-memory backend implements both `RaftLogStorage` and `RaftStateMachine` using RAM-backed structures. It provides no persistence and is suitable only for testing and demos, as explicitly noted in the example READMEs.

### RocksDB Backend (RocksStore)

The `examples/rocksstore/` directory contains a production-ready implementation:
- [`src/log_store.rs`](https://github.com/databendlabs/openraft/blob/main/src/log_store.rs): Persistent `RaftLogStorage` using RocksDB with WAL and `sync` options for durability.
- [`src/state_machine.rs`](https://github.com/databendlabs/openraft/blob/main/src/state_machine.rs): Persistent `RaftStateMachine` with snapshot support.

This backend guarantees durability through RocksDB's write-ahead log and is the recommended pattern for production deployments.

### OpenDAL Integration

For remote object storage (S3, OSS, etc.), OpenRaft supports `OpenDalStore`, where logs remain local (e.g., RocksDB) while snapshots are uploaded via OpenDAL. Snapshot durability then depends on the remote storage's consistency model.

## Implementing a Custom Storage Backend

To create a production storage layer, implement the traits with strict adherence to the durability and ordering constraints.

### Minimal In-Memory Log Store

The following skeleton demonstrates the required structure for `RaftLogStorage`, satisfying the async, `Send + Sync`, and callback requirements:

```rust
use openraft::storage::RaftLogStorage;
use openraft::{RaftTypeConfig, LogState, Vote, LogId};

pub struct MemLogStore<C: RaftTypeConfig> {
    pub logs: Vec<C::Entry>,
    pub vote: Option<Vote<C::NodeId>>,
}

#[openraft::async_trait]
impl<C> RaftLogStorage<C> for MemLogStore<C>
where
    C: RaftTypeConfig,
{
    type LogReader = Self;

    async fn get_log_state(&mut self) -> std::io::Result<LogState<C>> {
        let last = self.logs.last().map(|e| e.log_id().clone());
        Ok(LogState {
            last_purged_log_id: None,
            last_log_id: last,
        })
    }

    async fn get_log_reader(&mut self) -> Self::LogReader {
        self.clone()
    }

    async fn save_vote(&mut self, vote: &Vote<C::NodeId>) -> std::io::Result<()> {
        self.vote = Some(vote.clone());
        Ok(())
    }

    async fn append<I>(&mut self, entries: I, callback: openraft::storage::IOFlushed<C>) -> std::io::Result<()>
    where
        I: IntoIterator<Item = C::Entry> + openraft::OptionalSend,
        I::IntoIter: openraft::OptionalSend,
    {
        self.logs.extend(entries);
        callback();
        Ok(())
    }

    async fn truncate_after(&mut self, last_log_id: Option<LogId<C::NodeId>>) -> std::io::Result<()> {
        if let Some(id) = last_log_id {
            self.logs.retain(|e| e.log_id() <= &id);
        } else {
            self.logs.clear();
        }
        Ok(())
    }

    async fn purge(&mut self, log_id: LogId<C::NodeId>) -> std::io::Result<()> {
        self.logs.retain(|e| e.log_id() > &log_id);
        Ok(())
    }
}

```

Key implementation details:
- `append` calls the **callback** after entries are logically stored.
- All methods respect the **no-hole** rule by maintaining contiguous indices.
- The trait bounds (`OptionalSend`, `OptionalSync`) are satisfied automatically.

### RocksDB-Backed Log Store Skeleton

For production durability, implement `RaftLogStorage` using RocksDB with explicit `fsync`:

```rust
use openraft::{RaftLogStorage, RaftTypeConfig, LogState, Vote, LogId};
use rocksdb::{DB, WriteBatch};

pub struct RocksLogStore<C: RaftTypeConfig> {
    db: DB,
    _t: std::marker::PhantomData<C>,
}

#[openraft::async_trait]
impl<C> RaftLogStorage<C> for RocksLogStore<C>
where
    C: RaftTypeConfig,
{
    type LogReader = Self;

    async fn get_log_state(&mut self) -> std::io::Result<LogState<C>> {
        unimplemented!()
    }

    async fn get_log_reader(&mut self) -> Self::LogReader {
        self.clone()
    }

    async fn save_vote(&mut self, vote: &Vote<C::NodeId>) -> std::io::Result<()> {
        self.db.put(b"vote", bincode::serialize(vote)?)?;
        self.db.flush()?; // fsync for durability
        Ok(())
    }

    async fn append<I>(&mut self, entries: I, callback: openraft::storage::IOFlushed<C>) -> std::io::Result<()>
    where
        I: IntoIterator<Item = C::Entry> + openraft::OptionalSend,
        I::IntoIter: openraft::OptionalSend,
    {
        let mut batch = WriteBatch::default();
        for e in entries {
            let key = format!("log:{:020}", e.log_id().index);
            batch.put(key.as_bytes(), bincode::serialize(&e)?);
        }
        self.db.write(batch)?;
        self.db.flush()?; // Guarantee durability before callback
        callback();
        Ok(())
    }
}

```

Key production requirements:
- Calls `db.flush()` (or equivalent `sync`) to meet the **durability** requirement.
- Uses **bincode** (or any serializer) to store binary entries.
- Implements the same **no-hole** guarantee by using a contiguous key space (`log:<index>`).

### Snapshot Builder Implementation

The `RaftSnapshotBuilder` trait creates consistent snapshots:

```rust
use openraft::{RaftSnapshotBuilder, RaftTypeConfig, Snapshot, SnapshotMeta};

pub struct MySnapshotBuilder<C: RaftTypeConfig> {
    db: rocksdb::DB,
    _t: std::marker::PhantomData<C>,
}

#[openraft::async_trait]
impl<C> RaftSnapshotBuilder<C> for MySnapshotBuilder<C>
where
    C: RaftTypeConfig,
{
    async fn build_snapshot(&mut self) -> std::io::Result<Snapshot<C>> {
        let meta = SnapshotMeta {
            last_log_id: /* fetch from log store */,
            last_membership: /* fetch from state machine */,
            ..Default::default()
        };
        
        let tmp = tempfile::NamedTempFile::new()?;
        // ... copy DB contents to temp file ...
        
        Ok(Snapshot {
            meta,
            data: std::fs::File::open(tmp.path())?,
        })
    }
}

```

Critical implementation notes:
- The snapshot builder must **capture a consistent view** of both the log and the state machine.
- The produced `Snapshot` is later installed via `RaftStateMachine::install_snapshot`.

### Wiring Storage into a Raft Instance

Finally, instantiate the Raft node with your storage backend:

```rust
use openraft::{Raft, Config, declare_raft_types};

declare_raft_types! {
    pub TypeConfig:
        D = MyCommand,
        R = MyResponse;
}

let log_store = RocksLogStore::<TypeConfig>::new("path/to/db")?;
let state_machine = MyStateMachine::new(...);
let snapshot_builder = MySnapshotBuilder::new(log_store.db.clone());

let raft = Raft::new(
    node_id,
    Arc::new(Config::default()),
    MyNetwork, // implements RaftNetworkV2
    log_store,
    Arc::new(state_machine),
).await?;

```

This wiring pattern mirrors the production `rocksstore` example in the repository ([`examples/rocksstore/src/test.rs`](https://github.com/databendlabs/openraft/blob/main/examples/rocksstore/src/test.rs)), demonstrating the required integration of the storage backend with the Raft core.

## Summary

Implementing a storage backend for OpenRaft requires strict adherence to architectural constraints defined in the `openraft/src/storage/v2/` module:

- **Implement two core traits**: `RaftLogStorage` for log and vote persistence, and `RaftStateMachine` for state machine application and snapshots.
- **Guarantee serialization**: All write operations must execute sequentially without reordering, ensuring later writes never overtake earlier ones.
- **Maintain hole-free logs**: Log indices must remain contiguous; truncation and purge operations must not create gaps.
- **Ensure durability**: Writes must be durable (e.g., via `fsync` or database WAL) before returning, with `save_vote` persisting votes immediately and `append` invoking the `IOFlushed` callback only after persistence.
- **Satisfy thread safety**: Implementations must be `Send + Sync` and compatible with `OptionalSend` and `OptionalSync` bounds to function across async replication tasks.

## Frequently Asked Questions

### What are the two core traits required for an OpenRaft storage backend?

OpenRaft requires implementations of **`RaftLogStorage<C>`** and **`RaftStateMachine<C>`**, defined in [`openraft/src/storage/v2/raft_log_storage.rs`](https://github.com/databendlabs/openraft/blob/main/openraft/src/storage/v2/raft_log_storage.rs) and [`openraft/src/storage/v2/raft_state_machine.rs`](https://github.com/databendlabs/openraft/blob/main/openraft/src/storage/v2/raft_state_machine.rs) respectively. `RaftLogStorage` handles log entries, vote persistence, and committed log tracking, while `RaftStateMachine` manages business logic application, snapshot creation, and snapshot installation.

### Why must OpenRaft storage operations be serialized?

All write operations—including `append`, `truncate_after`, `purge`, and `save_vote`—must be **serialized** so that later writes never overtake earlier ones, as documented in the `RaftLogStorage` trait comments. This ordering guarantee ensures that the Raft core can rely on a consistent view of the log during leader election, log replication, and crash recovery, preventing split-brain scenarios and log divergence.

### How does OpenRaft ensure durability guarantees are met?

OpenRaft requires that once a write method returns, the data must be **durable** on stable storage, typically achieved through `fsync`, database write-ahead logs, or equivalent mechanisms. Specifically, `save_vote` must persist the vote before returning, and `append` must invoke the `IOFlushed` callback only after entries are confirmed on disk, as specified in the trait documentation for `RaftLogStorage` in [`openraft/src/storage/v2/raft_log_storage.rs`](https://github.com/databendlabs/openraft/blob/main/openraft/src/storage/v2/raft_log_storage.rs).

### Can I use an in-memory storage backend for production OpenRaft deployments?

No, the in-memory backend (`MemStore`) found in [`examples/raft-kv-memstore/src/store/mod.rs`](https://github.com/databendlabs/openraft/blob/main/examples/raft-kv-memstore/src/store/mod.rs) is explicitly designed for **testing and demonstrations only**, as noted in the example READMEs. Production deployments must use a persistent backend such as the RocksDB implementation in `examples/rocksstore/`, which guarantees durability through write-ahead logs and `sync` operations, ensuring data survives process crashes and restarts.