What is the Storage Interface in OpenRaft? A Complete Guide to the Storage Traits

The storage interface in OpenRaft is a set of Rust traits—RaftLogStorage, RaftLogReader, RaftStateMachine, and RaftSnapshotBuilder—that applications implement to provide pluggable persistence for logs, votes, and state machine snapshots.

OpenRaft, maintained by databendlabs/openraft, isolates all disk I/O and persistence concerns behind these storage traits. By implementing this interface, you enable the Raft consensus engine to append log entries, read them for replication, apply committed entries to your state machine, and manage snapshots—without the core library knowing anything about your underlying storage technology.

Core Storage Traits in the OpenRaft Interface

The storage interface is defined in the openraft::storage module and consists of four primary traits that handle distinct aspects of persistence.

RaftLogStorage: Persistent Log and Vote Handling

RaftLogStorage is the central trait for durable log storage. It manages the Raft log, persists the node's vote, and tracks the committed log index.

Key methods include:

• get_log_state() – Returns the last purged log ID and the last log ID currently stored.
• append() – Writes new entries to the log and invokes the IOFlushed callback once durable.
• truncate_after() – Removes entries after a specific log ID, typically during log conflict resolution.
• purge() – Permanently removes entries up to a specific log ID after they have been snapshotted.
• save_vote() – Persists the node's current vote (term and candidate).
• save_committed() and read_committed() – Persist and retrieve the highest committed log index.

Source: openraft/src/storage/v2/raft_log_storage.rs

RaftLogReader: Read-Only Access for Replication

RaftLogReader provides the read-only interface used by replication tasks to stream entries to followers. Implementations are typically returned by RaftLogStorage::get_log_reader().

Key methods include:

• try_get_log_entry() – Retrieves a single entry by index.
• read_log_entries() – Reads a range of entries between start and end log IDs.

Source: openraft/src/storage/v2/raft_log_reader.rs

RaftStateMachine: Applying Committed Entries

RaftStateMachine is where your application logic lives. After the Raft consensus layer commits a log entry, it calls apply() on your state machine implementation.

Key methods include:

• apply() – Receives a slice of committed entries and applies them to the application state.
• snapshot() – Creates a point-in-time snapshot of the current state machine state.
• restore_snapshot() – Restores the state machine from a previously created snapshot.

Source: openraft/src/storage/v2/raft_state_machine.rs

RaftSnapshotBuilder: Incremental Snapshot Construction

RaftSnapshotBuilder provides an optional interface for building snapshots incrementally, which is particularly useful when state machine data is too large to fit in memory.

Key methods include:

• begin_snapshot() – Initiates the snapshot process and returns metadata.
• build_snapshot() – Writes the snapshot data to the provided async writer.

Source: openraft/src/storage/v2/raft_snapshot_builder.rs

How the Storage Interface Works in Practice

Understanding the interaction between these traits clarifies how OpenRaft uses your storage implementation.

Log Replication Flow

When the Raft node receives client requests, the core engine calls RaftLogStorage::append() to persist entries. Once flushed, the IOFlushed callback notifies the Raft core that entries are safe to replicate. The core then obtains a RaftLogReader via get_log_reader() to stream entries to follower nodes.

Commit and Apply Flow

Once a log entry is committed (acknowledged by a quorum), the leader calls RaftStateMachine::apply() with the committed entries. Your implementation updates the application state accordingly. The storage layer may also persist the committed index via save_committed() to survive restarts.

Snapshotting Flow

When the log grows too large, the Raft core initiates snapshotting. It calls RaftStateMachine::snapshot() for simple cases, or uses RaftSnapshotBuilder for streaming large datasets. After snapshot completion, the core calls RaftLogStorage::purge() to remove obsolete log entries.

Implementing the Storage Interface: Minimal Example

Below is a minimal in-memory implementation illustrating how to satisfy the RaftLogStorage and RaftLogReader traits. This pattern is suitable for testing or lightweight applications.

use openraft::storage::{RaftLogStorage, RaftLogReader, IOFlushed, LogState};
use openraft::{LogId, LogIdOf, RaftTypeConfig, VoteOf};
use std::collections::BTreeMap;
use std::io;

pub struct SimpleLogStore<C>
where
    C: RaftTypeConfig,
{
    log: BTreeMap<u64, C::Entry>,
    last_purged: Option<LogId<C>>,
}

#[async_trait::async_trait]
impl<C> RaftLogStorage<C> for SimpleLogStore<C>
where
    C: RaftTypeConfig,
{
    type LogReader = Self;

    async fn get_log_state(&mut self) -> Result<LogState<C>, io::Error> {
        let last = self.log.keys().next_back().cloned().map(|i| LogId::new(0, i));
        Ok(LogState {
            last_purged_log_id: self.last_purged,
            last_log_id: last,
        })
    }

    async fn get_log_reader(&mut self) -> Self::LogReader {
        self.clone()
    }

    async fn save_vote(&mut self, _vote: &VoteOf<C>) -> Result<(), io::Error> {
        Ok(())
    }

    async fn append<I>(&mut self, entries: I, callback: IOFlushed<C>) -> Result<(), io::Error>
    where
        I: IntoIterator<Item = C::Entry> + openraft::OptionalSend,
    {
        for entry in entries {
            let idx = entry.log_id().index();
            self.log.insert(idx, entry);
        }
        callback.io_completed(Ok(()));
        Ok(())
    }

    async fn truncate_after(&mut self, last_log_id: Option<LogIdOf<C>>) -> Result<(), io::Error> {
        let keep = last_log_id.map(|id| id.index() + 1).unwrap_or(0);
        self.log = self.log.split_off(&keep);
        Ok(())
    }

    async fn purge(&mut self, log_id: LogIdOf<C>) -> Result<(), io::Error> {
        self.log = self.log.split_off(&(log_id.index() + 1));
        self.last_purged = Some(log_id);
        Ok(())
    }
}

#[async_trait::async_trait]
impl<C> RaftLogReader<C> for SimpleLogStore<C>
where
    C: RaftTypeConfig,
{
    async fn get_log_state(&mut self) -> Result<LogState<C>, io::Error> {
        <Self as RaftLogStorage<C>>::get_log_state(self).await
    }

    async fn try_get_log_entry(&mut self, index: u64) -> Result<Option<C::Entry>, io::Error> {
        Ok(self.log.get(&index).cloned())
    }

    async fn read_log_entries(
        &mut self,
        start: LogIdOf<C>,
        end: LogIdOf<C>,
    ) -> Result<Vec<C::Entry>, io::Error> {
        Ok(self.log.range(start.index()..=end.index()).map(|(_, v)| v.clone()).collect())
    }
}

Critical implementation details from the source code:

• append must invoke the IOFlushed callback to signal durability.
• get_log_state returns both last_purged_log_id and last_log_id to establish the valid log range.
• The same type can implement both RaftLogStorage and RaftLogReader when the storage supports concurrent reads.

Using the Built-In MemStore Implementation

For testing and development, OpenRaft provides a complete in-memory storage implementation in the memstore crate. This demonstrates production-quality trait implementations.

use openraft::memstore::{MemLogStore, MemStateMachine, TypeConfig};
use openraft::{Raft, Config};
use std::sync::Arc;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    // MemLogStore implements RaftLogStorage<TypeConfig>
    let log_store = Arc::new(MemLogStore::default());
    
    // MemStateMachine implements RaftStateMachine<TypeConfig>
    let state_machine = Arc::new(MemStateMachine::default());

    // Initialize the Raft node with the storage interface implementations
    let raft = Raft::new(0, Config::default(), log_store, state_machine).await?;
    
    Ok(())
}

The MemLogStore implementation resides in stores/memstore/src/lib.rs and provides thread-safe, atomic operations suitable for concurrent access by the Raft core and replication tasks.

Summary

The storage interface in OpenRaft decouples the consensus engine from persistence mechanisms through a trait-based abstraction layer. Key takeaways include:

• Four core traits define the interface: RaftLogStorage for log persistence, RaftLogReader for replication reads, RaftStateMachine for applying committed entries, and RaftSnapshotBuilder for snapshot creation.
• Location: All traits are defined in openraft/src/storage/v2/ and re-exported from openraft::storage.
• Critical contract: Implementations must maintain hole-free logs, invoke IOFlushed callbacks after durable writes, and handle concurrent access safely.
• Reference implementation: The memstore crate provides a complete in-memory implementation suitable for testing and as a template for production stores.

Frequently Asked Questions

What is the difference between RaftLogStorage and RaftLogReader?

RaftLogStorage is the write-path interface used by the Raft core to append entries, truncate logs, and persist votes. RaftLogReader is a read-only view typically used by replication tasks to stream entries to followers. While a single type can implement both traits, separating them allows the storage implementation to optimize read and write paths independently, such as using read replicas for replication traffic.

How does OpenRaft ensure data durability through the storage interface?

Durability is enforced through the IOFlushed callback mechanism. When the Raft core calls RaftLogStorage::append(), it passes a callback object. The storage implementation must write entries to durable media (disk, SSD, replicated storage) and then invoke callback.io_completed(Ok(())) before returning. This synchronous acknowledgment ensures that the Raft core only considers entries committed after they are durably stored, preventing data loss during crashes.

Can I use a custom state machine with the OpenRaft storage interface?

Yes, the RaftStateMachine trait is designed specifically for application-defined state machines. You implement apply() to receive committed log entries and update your application state, snapshot() to serialize your state, and restore_snapshot() to recover from snapshots. This design allows OpenRaft to manage the consensus and replication while your code handles business logic, whether that is a key-value store, a database, or a custom application.

Where can I find a complete example implementation of the storage interface?

The stores/memstore/ directory in the OpenRaft repository contains MemLogStore and MemStateMachine, which provide complete, production-quality implementations of all storage traits. These implementations demonstrate proper handling of concurrent access, atomic operations, and the IOFlushed callback pattern. Additionally, the test suite in tests/tests/public_api_test.rs shows how to instantiate and use these implementations with the Raft struct.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}