How OpenRaft Handles State Snapshots: Policy-Driven Log Compaction

OpenRaft handles state snapshots through a policy-driven, asynchronous pipeline that compacts the Raft log by triggering snapshot builds via the RaftSnapshotBuilder trait, running them in background tasks, and installing the resulting snapshots on followers or after restarts.

OpenRaft's snapshot subsystem provides a pluggable mechanism for log compaction and state machine persistence. Understanding how OpenRaft handles state snapshots is essential for configuring optimal performance and ensuring data durability in distributed consensus deployments.

Configuring Snapshot Policies in OpenRaft

Snapshot behavior is governed by the Config::snapshot_policy field, which determines when the engine may initiate a new snapshot build.

The SnapshotPolicy Enum

The policy is defined in openraft/src/config/config.rs (lines 29-38):

pub enum SnapshotPolicy {
    /// Build a snapshot after the log has grown `threshold` entries since the last snapshot.
    LogsSinceLast(u64),
    /// Never trigger automatically; the application must call `Raft::trigger().snapshot()`.
    Never,
}

Policy Evaluation Logic

The core evaluates the policy in RaftCore::trigger_routine_actions (lines 68-77 of openraft/src/core/raft_core.rs):

if let Some(at) = self
    .config
    .snapshot_policy
    .should_snapshot(&self.engine.state, self.core_state.snapshot_tried_at.as_ref())
{
    self.core_state.snapshot_tried_at = Some(at);
    self.trigger_snapshot();
}

The should_snapshot method (lines 50-60 of openraft/src/config/config.rs) compares the committed log index against the last snapshot index plus the configured threshold.

Triggering Snapshot Jobs in the OpenRaft Engine

When the policy conditions are met, RaftCore::trigger_snapshot forwards the request to the engine's SnapshotHandler (lines 50-55 of openraft/src/core/raft_core.rs):

pub(crate) fn trigger_snapshot(&mut self) {
    self.engine.snapshot_handler().trigger_snapshot();
}

The SnapshotHandler::trigger_snapshot implementation (lines 28-44 of openraft/src/engine/handler/snapshot_handler.rs) ensures only one build job runs at a time and pushes a command:

if self.state.io_state_mut().building_snapshot() {
    return false;
}
self.state.io_state.set_building_snapshot(true);
self.output.push_command(Command::from(sm::Command::build_snapshot()));
true

This Command::build_snapshot is later consumed by the state-machine worker to initiate the actual build process.

The RaftSnapshotBuilder Trait Interface

The contract for snapshot construction is defined by the RaftSnapshotBuilder trait in openraft/src/storage/v2/raft_snapshot_builder.rs (lines 20-33):

#[add_async_trait]
pub trait RaftSnapshotBuilder<C>: OptionalSend + OptionalSync + 'static
where C: RaftTypeConfig
{
    async fn build_snapshot(&mut self) -> Result<Snapshot<C>, io::Error>;
}

Implementations of this trait are responsible for serializing the state machine, writing the snapshot data, and returning a Snapshot containing the metadata and data stream.

State Machine Integration for OpenRaft Snapshots

The RaftStateMachine trait defines how the core interacts with the state machine for snapshot operations. Located in openraft/src/storage/v2/raft_state_machine.rs (lines 31-38), it declares:

type SnapshotBuilder: RaftSnapshotBuilder<C>;

async fn try_create_snapshot_builder(&mut self, force: bool) -> Option<Self::SnapshotBuilder> {
    // default: forward to the older `get_snapshot_builder`
    Some(self.get_snapshot_builder().await)
}
async fn get_snapshot_builder(&mut self) -> Self::SnapshotBuilder;

The try_create_snapshot_builder method allows the state machine to refuse snapshot creation if it is not ready, while get_snapshot_builder returns the actual builder instance.

Asynchronous Snapshot Construction in OpenRaft

The state-machine worker (sm::worker) processes the build command in openraft/src/core/sm/worker.rs (lines 55-71):

let builder = self.state_machine.try_create_snapshot_builder(false).await;
let Some(mut builder) = builder else {
    // State machine refused to build snapshot now.
    let res = CommandResult::new(Ok(Response::BuildSnapshotDone(None)));
    resp_tx.send(Notification::sm(res)).await.ok();
    return;
};

let _handle = C::spawn(async move {
    let res = builder.build_snapshot().await.sto_write_snapshot(None);
    let res = res.map(|snap| Response::BuildSnapshotDone(Some(snap.meta)));
    let cmd_res = CommandResult::new(res);
    resp_tx.send(Notification::sm(cmd_res)).await.ok();
});

The builder runs in a spawned asynchronous task to avoid blocking the core. Upon completion, the snapshot metadata is returned to the core via SnapshotHandler::update_snapshot.

Installing Snapshots on Followers and Restarts

When a follower receives a snapshot via replication or when a node restarts, the core invokes RaftStateMachine::install_snapshot. The worker calls this method in openraft/src/core/sm/worker.rs (lines 74-81):

self.state_machine.install_snapshot(&meta, snapshot).await?;

Implementations must replace the state machine state with the snapshot contents, persist the data, and remove obsolete log entries. After successful installation, the core updates its snapshot_last_log_id and truncates the log up to the snapshot index.

Manually Triggering State Snapshots in OpenRaft

While automatic snapshots rely on the SnapshotPolicy, applications can force immediate snapshot creation using the trigger API:

use openraft::Raft;
use openraft::RaftMetrics;

async fn manual_snapshot<NodeId, MyApp>(
    raft: &Raft<NodeId, MyApp>
) -> anyhow::Result<()>
where
    NodeId: openraft::NodeId,
    MyApp: openraft::RaftTypeConfig,
{
    // Force a snapshot regardless of the policy.
    raft.trigger().snapshot().await?;
    
    // Wait until the snapshot is built and persisted.
    let metrics: RaftMetrics<NodeId> = raft.metrics().await;
    println!("last snapshot log id: {:?}", metrics.last_snapshot);
    Ok(())
}

This call chain enters RaftCore::trigger_snapshot, bypassing the policy check and immediately enqueueing a Command::build_snapshot as described in the engine handler flow.

Summary

• Policy-driven activation: OpenRaft uses SnapshotPolicy (LogsSinceLast or Never) to determine when to initiate snapshot builds, evaluated in RaftCore::trigger_routine_actions.
• Asynchronous construction: The RaftSnapshotBuilder trait defines the build interface, executed in a spawned task by the state-machine worker (sm::worker) to prevent blocking the Raft core.
• Command-based coordination: Snapshot jobs are triggered via Command::build_snapshot, handled by SnapshotHandler::trigger_snapshot, ensuring only one build runs concurrently.
• State machine integration: The RaftStateMachine trait provides try_create_snapshot_builder and install_snapshot, allowing custom implementations to control build readiness and apply snapshots during replication or restarts.
• Manual override: Applications can bypass automatic policies using raft.trigger().snapshot() for on-demand log compaction.

Frequently Asked Questions

How does OpenRaft decide when to create a snapshot automatically?

OpenRaft evaluates the SnapshotPolicy configured in Config::snapshot_policy during routine core checks. If the policy is LogsSinceLast(threshold), the engine compares the committed log index against the last snapshot index plus the threshold in SnapshotPolicy::should_snapshot. When the condition is met, RaftCore::trigger_snapshot enqueues a build command. If the policy is Never, automatic triggering is disabled and the application must initiate snapshots manually.

What is the difference between try_create_snapshot_builder and get_snapshot_builder?

try_create_snapshot_builder is the primary entry point that allows the state machine to refuse snapshot creation gracefully by returning None if it is not ready, such as during heavy write loads. It has a default implementation that delegates to get_snapshot_builder. The latter, get_snapshot_builder, must return a builder instance unconditionally and is the method implementors typically override to provide their RaftSnapshotBuilder implementation.

Can snapshot builds run concurrently with normal Raft operations?

Yes. OpenRaft explicitly spawns snapshot construction as an asynchronous task using C::spawn in the state-machine worker (sm::worker). This ensures that build_snapshot runs independently without blocking the Raft core's message processing or log replication. The engine tracks the build status via IOState::building_snapshot to prevent multiple concurrent builds of the same state machine.

How does a follower apply a received snapshot?

When a follower receives an InstallSnapshot RPC, the core eventually calls RaftStateMachine::install_snapshot through the state-machine worker. The implementation must replace the current state machine state with the snapshot contents, persist the data to durable storage, and remove obsolete log entries. After successful installation, the core updates its snapshot_last_log_id and truncates the log up to the snapshot index, ensuring consistency with the leader's state.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}