# Implementing a Custom Storage Backend for OpenRaft: A Complete Guide

> Implement a custom storage backend for OpenRaft by fulfilling the RaftLogStorage and RaftStateMachine traits. Ensure atomic persistence for robust distributed systems.

- Repository: [Databend Labs/openraft](https://github.com/databendlabs/openraft)
- Tags: how-to-guide
- Published: 2026-02-28

---

**To implement a custom storage backend for OpenRaft, you must implement the `RaftLogStorage` and `RaftStateMachine` traits from the `openraft` crate, ensuring strict serialization guarantees and atomic persistence of log entries, votes, and state machine snapshots.**

OpenRaft, the Raft consensus implementation maintained by databendlabs, separates the consensus algorithm from persistence concerns through well-defined storage traits. This architecture allows you to plug in any durable storage solution—whether RocksDB, PostgreSQL, S3, or a custom in-memory store—by implementing two core interfaces. This guide walks through the concrete steps required to build a production-ready custom storage backend for OpenRaft, referencing the actual source files and trait definitions from the repository.

## Understanding the Storage Architecture

OpenRaft defines two primary storage traits that decouple consensus logic from data persistence. According to the source code in `openraft/src/storage/v2/`, these contracts are:

- **`RaftLogStorage`** ([`raft_log_storage.rs`](https://github.com/databendlabs/openraft/blob/main/raft_log_storage.rs)): Manages the replicated log, persistent voting state, and optional committed index pointers.
- **`RaftStateMachine`** ([`raft_state_machine.rs`](https://github.com/databendlabs/openraft/blob/main/raft_state_machine.rs)): Holds application-specific state and applies committed log entries.

A third trait, `RaftLogReader`, is required for streaming log entries to replication tasks. The reference implementation in [`stores/memstore/src/lib.rs`](https://github.com/databendlabs/openraft/blob/main/stores/memstore/src/lib.rs) demonstrates a complete in-memory backend that satisfies all three contracts.

## Step 1: Define Your Type Configuration

Before implementing storage, you must define a **type configuration** that describes the request/response payloads, node identifier, and leader-id type. OpenRaft uses the `declare_raft_types!` macro to generate the necessary boilerplate.

```rust
use openraft::declare_raft_types;

declare_raft_types! {
    /// Your concrete configuration.
    pub MyConfig:
        D = MyRequest,          // client request type
        R = MyResponse,         // client response type
        Node = u64,             // node identifier (could be UUID, string, etc.)
        LeaderId = openraft::impls::leader_id_std::LeaderId<MyConfig>; // or custom
}

```

This pattern appears in [`stores/memstore/src/lib.rs`](https://github.com/databendlabs/openraft/blob/main/stores/memstore/src/lib.rs) and establishes the generic parameters used throughout your storage implementation.

## Step 2: Implement the RaftLogStorage Trait

The `RaftLogStorage` trait in [`openraft/src/storage/v2/raft_log_storage.rs`](https://github.com/databendlabs/openraft/blob/main/openraft/src/storage/v2/raft_log_storage.rs) requires you to persist log entries, votes, and purge pointers. Your implementation must guarantee **strict serialization**—all writes must complete in order, and later writes must not complete before earlier ones.

```rust
use openraft::{
    storage::{IOFlushed, RaftLogReader, RaftLogStorage},
    type_config::RaftTypeConfig,
    LogState, VoteOf, LogIdOf,
};
use std::io;

pub struct MyLogStore {
    // e.g., rocksdb::DB, sled::Db, etc.
    db: MyDb,
}

#[async_trait::async_trait]
impl<C> RaftLogStorage<C> for MyLogStore
where
    C: RaftTypeConfig,
{
    type LogReader = MyLogReader; // must implement `RaftLogReader<C>`

    async fn get_log_state(&mut self) -> Result<LogState<C>, io::Error> {
        // Read last_purged_log_id and last_log_id from storage.
        // Return `LogState { last_purged_log_id, last_log_id }`
        todo!()
    }

    async fn get_log_reader(&mut self) -> Self::LogReader {
        MyLogReader::new(self.db.clone())
    }

    async fn save_vote(&mut self, vote: &VoteOf<C>) -> Result<(), io::Error> {
        // Persist `vote` atomically.
        todo!()
    }

    async fn append<I>(&mut self, entries: I, callback: IOFlushed<C>) -> Result<(), io::Error>
    where
        I: IntoIterator<Item = C::Entry> + OptionalSend,
        I::IntoIter: OptionalSend,
    {
        // Serialize each entry and write to `db`.
        // After flushing to disk, invoke `callback.flushed()`.
        todo!()
    }

    async fn truncate_after(&mut self, last_log_id: Option<LogIdOf<C>>) -> Result<(), io::Error> {
        // Delete entries > `last_log_id`.
        todo!()
    }

    async fn purge(&mut self, log_id: LogIdOf<C>) -> Result<(), io::Error> {
        // Delete entries ≤ `log_id` and update `last_purged_log_id`.
        todo!()
    }
}

```

**Critical correctness guarantees** from the trait documentation:

- **No gaps in the log** – truncation must leave a continuous sequence.
- **Strict write serialization** – a later write must not complete before an earlier one.
- **Durable callback invocation** – the `callback` passed to `append` must only be invoked **after** entries are flushed to stable storage.

### Implementing the RaftLogReader

The `RaftLogReader` trait requires methods for streaming log entries to replication tasks. Define a separate reader struct that shares access to your underlying database:

```rust
pub struct MyLogReader {
    db: MyDb,
}

impl<C> RaftLogReader<C> for MyLogReader
where
    C: RaftTypeConfig,
{
    async fn get_log_entries(&mut self, start: u64, limit: Option<u64>) -> Result<Vec<C::Entry>, io::Error> {
        // Fetch entries from `start` up to `limit`.
        todo!()
    }

    async fn get_log_state(&mut self) -> Result<LogState<C>, io::Error> {
        // Same as `RaftLogStorage::get_log_state`, but read‑only.
        todo!()
    }
}

```

The `MemStore` reader in [`stores/memstore/src/lib.rs`](https://github.com/databendlabs/openraft/blob/main/stores/memstore/src/lib.rs) demonstrates how to share an underlying data structure between the storage and reader implementations.

## Step 3: Implement the RaftStateMachine Trait

The `RaftStateMachine` trait in [`openraft/src/storage/v2/raft_state_machine.rs`](https://github.com/databendlabs/openraft/blob/main/openraft/src/storage/v2/raft_state_machine.rs) manages your application state. It must apply committed log entries and handle snapshot generation and installation.

```rust
use openraft::{
    storage::{Snapshot, RaftSnapshotBuilder, RaftStateMachine, RaftStateMachineApplier},
    type_config::RaftTypeConfig,
    StorageError,
};

pub struct MyStateMachine {
    // e.g., in‑memory hash map, RocksDB column family, etc.
    inner: MyDb,
}

#[async_trait::async_trait]
impl<C> RaftStateMachine<C> for MyStateMachine
where
    C: RaftTypeConfig,
{
    async fn apply(&mut self, applier: &mut impl RaftStateMachineApplier<C>) -> Result<(), StorageError<C>> {
        // Iterate over `applier.entries()` and update `inner` accordingly.
        // Update your own `last_applied_log` and `last_membership`.
        todo!()
    }

    async fn get_snapshot_builder(&mut self) -> Result<Box<dyn RaftSnapshotBuilder<C>>, StorageError<C>> {
        // Return a builder that can serialize the current state into a snapshot.
        todo!()
    }

    async fn install_snapshot(&mut self, snapshot: &mut impl SnapshotData<C>) -> Result<(), StorageError<C>> {
        // Replace current state with `snapshot` data.
        todo!()
    }
}

```

If your state machine persists the committed index itself, you can omit `save_committed` and `read_committed` from your log store implementation.

## Step 4: Wire Everything Together

With your storage traits implemented, instantiate the `Raft` struct using your custom backend. This initialization pattern appears in the repository’s integration tests such as [`tests/tests/public_api_test.rs`](https://github.com/databendlabs/openraft/blob/main/tests/tests/public_api_test.rs).

```rust
use openraft::{Raft, Config};

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let config = Config::build("my-node".into())
        .validate()
        .unwrap();

    let log_store = MyLogStore::new(/* db handle */);
    let state_machine = MyStateMachine::new(/* db handle */);

    // Create the Raft node.
    let raft = Raft::new(config, log_store, state_machine).await?;

    // Use `raft` to join a cluster, propose commands, etc.
    Ok(())
}

```

## Step 5: Validate with the Test Suite

OpenRaft provides a **store builder** in [`openraft/src/testing/log/store_builder.rs`](https://github.com/databendlabs/openraft/blob/main/openraft/src/testing/log/store_builder.rs) that runs the full test matrix against your implementation. This validates that your backend satisfies the strict correctness contracts required for consensus safety.

```rust
use openraft::testing::log::store_builder::StoreBuilder;

#[tokio::test]
async fn custom_backend_passes_all_tests() {
    let builder = StoreBuilder::new(|| async {
        // Return a tuple (log_store, state_machine).
        (MyLogStore::new(...), MyStateMachine::new(...))
    });
    builder.run_all().await.unwrap();
}

```

Running this suite verifies that your implementation maintains **no holes in the log**, **serializes writes correctly**, and **invokes IO callbacks only after durable flushing**.

## Summary

Implementing a custom storage backend for OpenRaft requires careful adherence to the trait contracts defined in the `openraft` crate. The essential steps include:

- **Defining a `TypeConfig`** using the `declare_raft_types!` macro to establish your application’s request/response types and node identifiers.
- **Implementing `RaftLogStorage`** in [`openraft/src/storage/v2/raft_log_storage.rs`](https://github.com/databendlabs/openraft/blob/main/openraft/src/storage/v2/raft_log_storage.rs) to persist log entries, votes, and purge pointers with strict serialization guarantees.
- **Providing a `RaftLogReader`** implementation for streaming log entries to replication tasks.
- **Implementing `RaftStateMachine`** in [`openraft/src/storage/v2/raft_state_machine.rs`](https://github.com/databendlabs/openraft/blob/main/openraft/src/storage/v2/raft_state_machine.rs) to apply committed entries and manage snapshots.
- **Validating with `StoreBuilder`** from [`openraft/src/testing/log/store_builder.rs`](https://github.com/databendlabs/openraft/blob/main/openraft/src/testing/log/store_builder.rs) to ensure compliance with consensus safety requirements.

By following these patterns demonstrated in [`stores/memstore/src/lib.rs`](https://github.com/databendlabs/openraft/blob/main/stores/memstore/src/lib.rs), you can integrate any durable storage solution into OpenRaft, giving you a flexible, production-ready consensus layer tailored to your infrastructure.

## Frequently Asked Questions

### What are the consistency requirements for a custom OpenRaft storage backend?

Your implementation must guarantee three critical properties derived from the trait documentation in [`raft_log_storage.rs`](https://github.com/databendlabs/openraft/blob/main/raft_log_storage.rs): **no gaps in the log** (truncation must leave a continuous sequence), **strict write serialization** (later writes must not complete before earlier ones), and **durable callback invocation** (the `IOFlushed` callback passed to `append` must only be invoked after entries are flushed to stable storage). Violating these contracts can lead to consensus failures or split-brain scenarios.

### Can I use an existing database like RocksDB or PostgreSQL as the storage backend?

Yes. The `RaftLogStorage` and `RaftStateMachine` traits are database-agnostic interfaces. You can wrap a `rocksdb::DB` instance, a PostgreSQL connection pool, or an S3 client in your struct and implement the required async methods. The reference implementation in [`stores/memstore/src/lib.rs`](https://github.com/databendlabs/openraft/blob/main/stores/memstore/src/lib.rs) demonstrates the structural pattern for wrapping external storage handles. Ensure your underlying database transactions satisfy the strict serialization guarantees required by the traits.

### How does OpenRaft handle storage errors during log replication?

Storage errors returned from `RaftLogStorage` methods (such as `append` or `save_vote`) are treated as fatal to the current operation. If `append` fails, you must not invoke the `IOFlushed` callback, and the error propagates upward to the Raft engine. The consensus layer expects that storage implementations handle transient errors internally (e.g., through retries) and only return errors for unrecoverable failures. Persistent storage failures will cause the node to step down or halt to prevent safety violations.

### What is the difference between RaftLogStorage and RaftStateMachine?

**`RaftLogStorage`**, defined in [`openraft/src/storage/v2/raft_log_storage.rs`](https://github.com/databendlabs/openraft/blob/main/openraft/src/storage/v2/raft_log_storage.rs), manages the **Write-Ahead Log (WAL)**—it stores raw log entries, the node's persistent voting state, and log purge pointers. It is concerned with durability and replication. **`RaftStateMachine`**, defined in [`openraft/src/storage/v2/raft_state_machine.rs`](https://github.com/databendlabs/openraft/blob/main/openraft/src/storage/v2/raft_state_machine.rs), manages **application state**—it applies committed log entries to your business logic and handles snapshot generation and installation. While the log storage is mandatory for consensus safety, the state machine is where your application logic lives. They are architecturally separate to allow log compaction (snapshots) to occur independently of log storage operations.