Where to Find Storage Module Files in OpenRaft: Complete Guide to the Storage Layer

OpenRaft's storage module files are located in openraft/src/storage/ and its v2/ subdirectory, containing traits like RaftLogStorage, RaftStateMachine, and RaftSnapshotBuilder that define the persistence interface.

The databendlabs/openraft repository organizes its persistence layer inside the openraft/src/storage package. This module houses the core async traits and data structures that applications must implement to provide durable log storage, snapshot handling, and state-machine interfaces for the Raft consensus algorithm.

Core Storage Module Location in OpenRaft

The storage layer entry point is openraft/src/storage/mod.rs. This file declares sub-modules (callback, helper, log_state, snapshot, and v2) and publicly re-exports the key types required by storage implementers.

According to the OpenRaft source code, the storage module serves as the public API boundary. When implementing a custom backend, you will primarily interact with symbols re-exported from this root module while referencing the concrete trait definitions located in the v2/ subdirectory.

Key Storage Files and Their Responsibilities

Entry Point and Public API (mod.rs)

Located at [openraft/src/storage/mod.rs](https://github.com/databendlabs/openraft/blob/main/openraft/src/storage/mod.rs), this file aggregates all storage-related symbols. It re-exports RaftLogStorage, RaftStateMachine, LogState, Snapshot, and helper utilities, providing a single import path for implementers.

Log State Tracking (log_state.rs)

The [openraft/src/storage/log_state.rs](https://github.com/databendlabs/openraft/blob/main/openraft/src/storage/log_state.rs) file defines LogState<C>, a structure that tracks last_purged_log_id and last_log_id. This bookkeeping structure is essential for the Raft engine to determine log boundaries and truncation points.

Snapshot Data Structures

Snapshot handling spans three files in the storage module root:

• [snapshot.rs](https://github.com/databendlabs/openraft/blob/main/openraft/src/storage/snapshot.rs) – Defines Snapshot<C>, which wraps a snapshot payload together with its metadata.
• [snapshot_meta.rs](https://github.com/databendlabs/openraft/blob/main/openraft/src/storage/snapshot_meta.rs) – Contains SnapshotMeta<C>, storing the last log ID and membership configuration associated with the snapshot.
• [snapshot_signature.rs](https://github.com/databendlabs/openraft/blob/main/openraft/src/storage/snapshot_signature.rs) – Implements SnapshotSignature for integrity verification of snapshot data.

Version 2 Storage Traits (v2/ Directory)

The concrete async trait definitions that storage backends must implement reside in the v2/ subdirectory. These traits define the contract between the Raft consensus engine and the persistence layer.

RaftLogStorage and RaftLogReader

Located in [openraft/src/storage/v2/raft_log_storage.rs](https://github.com/databendlabs/openraft/blob/main/openraft/src/storage/v2/raft_log_storage.rs), the RaftLogStorage<C> async trait defines methods including:

• get_log_state() – Returns current log boundaries
• append() – Writes new entries to the log
• purge() – Removes entries up to a specified log ID
• truncate_after() – Truncates the log after a given point
• save_vote() – Persists the current vote

The companion trait RaftLogReader<C> in [raft_log_reader.rs](https://github.com/databendlabs/openraft/blob/main/openraft/src/storage/v2/raft_log_reader.rs) provides try_get_log_entries() for fetching log entries and read_vote() for retrieving the persisted vote.

RaftStateMachine

The [openraft/src/storage/v2/raft_state_machine.rs](https://github.com/databendlabs/openraft/blob/main/openraft/src/storage/v2/raft_state_machine.rs) file defines RaftStateMachine<C>, which requires:

• apply() – Applies committed log entries to the state machine
• install_snapshot() – Replaces the state machine with a snapshot
• last_applied() – Returns the last applied log ID
• get_snapshot_builder() – Returns a builder for creating snapshots

RaftSnapshotBuilder

Found in [openraft/src/storage/v2/raft_snapshot_builder.rs](https://github.com/databendlabs/openraft/blob/main/openraft/src/storage/v2/raft_snapshot_builder.rs), the RaftSnapshotBuilder<C> trait defines build_snapshot(), which constructs a Snapshot<C> from the current state machine state.

Practical Implementation Example

The following example from the OpenRaft benchmark suite (benchmarks/minimal/src/store.rs) demonstrates a minimal, functional storage implementation. It shows how the traits discussed above are wired together in practice.

use std::collections::BTreeMap;
use std::io;
use std::ops::RangeBounds;
use std::sync::Arc;
use std::sync::atomic::{AtomicU64, Ordering};

use futures::Stream;
use openraft::storage::{
    EntryResponder, IOFlushed, LogState, RaftLogReader, RaftLogStorage,
    RaftSnapshotBuilder, RaftStateMachine, Snapshot,
};
use openraft::{alias::LogIdOf, Entry, SnapshotMeta, StoredMembership, Vote};

// Log Store implementation
pub struct LogStore {
    vote: tokio::sync::RwLock<Option<Vote<TypeConfig>>>,
    log: tokio::sync::RwLock<BTreeMap<u64, Entry<TypeConfig>>>,
    last_purged_log_id: tokio::sync::RwLock<Option<LogIdOf<TypeConfig>>>,
}

#[async_trait::async_trait]
impl RaftLogReader<TypeConfig> for Arc<LogStore> {
    async fn try_get_log_entries<RB>(
        &mut self,
        range: RB,
    ) -> Result<Vec<Entry<TypeConfig>>, io::Error>
    where
        RB: RangeBounds<u64> + Clone + std::fmt::Debug + Send,
    {
        let mut entries = vec![];
        let log = self.log.read().await;
        for (_, ent) in log.range(range) {
            entries.push(ent.clone());
        }
        Ok(entries)
    }

    async fn read_vote(&mut self) -> Result<Option<Vote<TypeConfig>>, io::Error> {
        Ok(*self.vote.read().await)
    }
}

#[async_trait::async_trait]
impl RaftLogStorage<TypeConfig> for Arc<LogStore> {
    type LogReader = Arc<LogStore>;

    async fn get_log_state(&mut self) -> Result<LogState<TypeConfig>, io::Error> {
        let last_purged = *self.last_purged_log_id.read().await;
        let log = self.log.read().await;
        let last_log = log.keys().last().cloned().map(|idx| LogId::new(0, idx));
        Ok(LogState {
            last_purged_log_id: last_purged,
            last_log_id: last_log,
        })
    }

    async fn get_log_reader(&mut self) -> Self::LogReader {
        Arc::clone(self)
    }

    async fn save_vote(&mut self, vote: &Vote<TypeConfig>) -> Result<(), io::Error> {
        *self.vote.write().await = Some(vote.clone());
        Ok(())
    }

    async fn append<I>(&mut self, entries: I, _callback: IOFlushed<TypeConfig>) -> Result<(), io::Error>
    where
        I: IntoIterator<Item = Entry<TypeConfig>> + Send,
    {
        let mut log = self.log.write().await;
        for ent in entries {
            log.insert(ent.log_id.index, ent);
        }
        Ok(())
    }

    async fn purge(&mut self, log_id: LogIdOf<TypeConfig>) -> Result<(), io::Error> {
        let mut log = self.log.write().await;
        log.retain(|idx, _| *idx > log_id.index);
        *self.last_purged_log_id.write().await = Some(log_id);
        Ok(())
    }

    async fn truncate_after(&mut self, last_log_id: Option<LogIdOf<TypeConfig>>) -> Result<(), io::Error> {
        let mut log = self.log.write().await;
        if let Some(last) = last_log_id {
            log.retain(|idx, _| *idx <= last.index);
        } else {
            log.clear();
        }
        Ok(())
    }
}

This example demonstrates the concrete implementation of RaftLogStorage and RaftLogReader using in-memory BTreeMap structures. The StateMachineStore (not fully shown here but referenced in the analysis) would similarly implement RaftStateMachine and RaftSnapshotBuilder to complete the storage layer.

Summary

• Primary Location: All storage module files in OpenRaft reside under openraft/src/storage/ with core trait definitions in the v2/ subdirectory.
• Entry Point: openraft/src/storage/mod.rs re-exports all public storage symbols including RaftLogStorage, RaftStateMachine, and Snapshot.
• Core Traits: The v2/ directory contains the async trait definitions that must be implemented: RaftLogStorage and RaftLogReader (in raft_log_storage.rs and raft_log_reader.rs), RaftStateMachine (in raft_state_machine.rs), and RaftSnapshotBuilder (in raft_snapshot_builder.rs).
• Data Structures: Supporting files like log_state.rs, snapshot.rs, and snapshot_meta.rs define the data structures used for log tracking and snapshot metadata.
• Implementation Pattern: Applications implement these traits to integrate backends like RocksDB or Sled, as demonstrated in the benchmark suite's store.rs example.

Frequently Asked Questions

Where is the main storage module entry point in OpenRaft?

The main entry point is openraft/src/storage/mod.rs. This file declares all storage sub-modules and publicly re-exports the essential types including RaftLogStorage, RaftStateMachine, LogState, and Snapshot. When implementing a custom storage backend, you typically import these types from openraft::storage while referencing the concrete trait definitions in the v2/ subdirectory.

What traits must I implement for a custom storage backend?

You must implement four core async traits defined in the v2/ directory: RaftLogStorage (defined in raft_log_storage.rs) for persisting log entries and votes; RaftLogReader (defined in raft_log_reader.rs) for reading log entries; RaftStateMachine (defined in raft_state_machine.rs) for applying committed entries and installing snapshots; and RaftSnapshotBuilder (defined in raft_snapshot_builder.rs) for creating snapshots from the state machine.

How does OpenRaft handle snapshot storage?

Snapshot storage is managed through the Snapshot struct (defined in snapshot.rs), SnapshotMeta (in snapshot_meta.rs), and SnapshotSignature (in snapshot_signature.rs). The RaftStateMachine trait provides install_snapshot() to replace the state machine with snapshot data, while RaftSnapshotBuilder defines build_snapshot() to serialize the current state machine into a Snapshot object. These structures track metadata including the last log ID and membership configuration associated with each snapshot.

What's the difference between the storage root and the v2 subdirectory?

The storage root directory (openraft/src/storage/) contains data structures and helper types like LogState and Snapshot that are used across the library. The v2/ subdirectory contains the actual async trait definitions (RaftLogStorage, RaftStateMachine, etc.) that constitute the current storage API. The "v2" naming indicates the second version of the storage interface, which uses async traits rather than the callback-based approach found in earlier versions of OpenRaft.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}