How Neo's Persistence Layer Works with Storage Providers and the StoreCache Mechanism

Neo's persistence layer uses a provider-based architecture where IStoreProvider implementations supply concrete storage engines, while StoreCache adds change-tracking and isolation on top of IStore or IStoreSnapshot instances, enabling atomic commits and read-through caching.

Neo’s persistence layer provides a flexible, provider-agnostic foundation for blockchain state management in the neo-project/neo repository. This architecture abstracts storage engines behind consistent interfaces, allowing developers to swap between in-memory stores for testing and persistent databases like LevelDB or RocksDB for production without modifying core blockchain logic.

Storage Provider Architecture

The persistence layer is built around three core concepts: Storage Providers that supply concrete implementations, Store/Snapshot abstractions that handle raw data access, and Cache Layers that provide isolation and batching.

The IStoreProvider Interface

Any storage engine must implement IStoreProvider to integrate with Neo. This interface defines a provider name and a factory method that returns an IStore instance for a given path.

public interface IStoreProvider
{
    string Name { get; }
    IStore GetStore(string? path);
}

Source: src/Neo/Persistence/IStoreProvider.cs

StoreFactory Registration

StoreFactory maintains a static registry of providers. The static constructor automatically registers the built-in in-memory provider and sets it as the default (keyed by empty string "").

static StoreFactory()
{
    var memProvider = new MemoryStoreProvider();
    RegisterProvider(memProvider);
    s_providers.Add("", memProvider);
}

Plugins register custom providers by calling StoreFactory.RegisterProvider(myProvider), making them available system-wide.

Source: src/Neo/Persistence/StoreFactory.cs

Built-in Memory Provider

The default MemoryStoreProvider returns a MemoryStore, which implements IStore using a thread-safe ConcurrentDictionary<byte[], byte[]>. This is ideal for unit tests and "no-persistence" modes.

Source: src/Neo/Persistence/Providers/MemoryStoreProvider.cs, MemoryStore.cs

Store and Snapshot Abstraction

The IStore Interface

IStore combines read-only and write capabilities. It extends IReadOnlyStore and IWriteStore, and exposes GetSnapshot() to create isolated views.

IStoreSnapshot GetSnapshot();

Source: src/Neo/Persistence/IStore.cs

IStoreSnapshot for Isolation

IStoreSnapshot represents a point-in-time view of the underlying storage. Writes are staged in a private batch and applied atomically via Commit(). This isolation prevents partial state updates during block validation.

Source: src/Neo/Persistence/IStoreSnapshot.cs, MemorySnapshot.cs

The StoreCache Mechanism

StoreCache is the concrete implementation that bridges high-level cache operations to the low-level storage interfaces. It inherits from DataCache and adds read-through and write-through capabilities.

DataCache Foundation

DataCache is an abstract class that maintains an in-memory change set (dictionary of added, updated, and deleted entries). It defines the public API (Add, Delete, Commit, Seek) and calls abstract protected methods (AddInternal, UpdateInternal, DeleteInternal, TryGetInternal) to interact with the actual storage.

Source: src/Neo/Persistence/DataCache.cs

StoreCache Implementation

StoreCache implements the abstract methods to forward operations to an IStore or IStoreSnapshot. It offers two constructors:

// Read-only or direct store access
public StoreCache(IStore store, bool readOnly = true) : base(readOnly)
{
    _store = store;
}

// Snapshot mode for transactional writes
public StoreCache(IStoreSnapshot snapshot) : base(false)
{
    _store = snapshot;
    _snapshot = snapshot;
}

Source: src/Neo/Persistence/StoreCache.cs

Read-Through and Write-Through Behavior

Read-through occurs when GetInternal or TryGetInternal cannot find an entry in the local change set. The cache forwards the request to _store.Get() or _store.TryGet(), then wraps the raw byte arrays in StorageItem objects.

Write-through (snapshot mode) stages changes in the snapshot's batch rather than immediately writing to disk. The AddInternal, UpdateInternal, and DeleteInternal methods call _snapshot.Put() and _snapshot.Delete():

protected override void AddInternal(StorageKey key, StorageItem value)
{
    _snapshot?.Put(key.ToArray(), value.ToArray());
}

Commit and Atomicity

StoreCache.Commit() implements a two-phase commit process:

1. Flush change set: base.Commit() calls AddInternal, UpdateInternal, and DeleteInternal for all tracked changes, writing them to the snapshot's batch.
2. Persist snapshot: _snapshot?.Commit() atomically writes the batch to the underlying storage.

public override void Commit()
{
    base.Commit();               // apply change set to snapshot
    _snapshot?.Commit();       // persist snapshot batch
}

If the cache is disposed without committing, the snapshot is discarded, rolling back uncommitted changes.

Working with Different Storage Providers

Using the Default In-Memory Store

For unit tests or ephemeral nodes, use the default memory provider without external dependencies:

// Create the default in-memory store (empty string selects the default)
var store = StoreFactory.GetStore("", "");

// Simple read/write via StoreCache
using var cache = new StoreCache(store);
var key = new StorageKey(new byte[] { 0x10 });
var value = new StorageItem(new byte[] { 0xFF });

cache.Add(key, value);        // stage write
cache.Commit();               // persisted to the in-memory dictionary

var retrieved = cache[key];   // read-through
Console.WriteLine(BitConverter.ToString(retrieved.GetSpan()));

Relevant files: StoreFactory.cs, MemoryStoreProvider.cs, MemoryStore.cs, StoreCache.cs.

Implementing a Custom Provider

To integrate RocksDB, LevelDB, or another engine, implement IStoreProvider and register it at startup:

public class RocksDbProvider : IStoreProvider
{
    public string Name => "rocksdb";

    public IStore GetStore(string? path)
    {
        // Assume RocksDbStore implements IStore
        return new RocksDbStore(path ?? "rocks.db");
    }
}

// Register at application startup
StoreFactory.RegisterProvider(new RocksDbProvider());

// Use by name
var rocksStore = StoreFactory.GetStore("rocksdb", "data/rocks");
using var cache = new StoreCache(rocksStore);
// ... normal cache usage ...

Relevant files: IStoreProvider.cs, StoreFactory.cs.

Integration in NeoSystem

NeoSystem exposes convenience properties that construct StoreCache instances on demand, abstracting the persistence layer from blockchain logic:

public StoreCache StoreView => new(_store);               // read-only view
public StoreCache GetSnapshot() => new(_store.GetSnapshot());
public StoreCache GetSnapshotCache() => new(_store.GetSnapshot());

These helpers are used throughout the node for block processing, RPC handlers, and smart-contract VM execution to obtain appropriately isolated cache instances.

Source: src/Neo/NeoSystem.cs

Summary

• Neo's persistence layer is provider-agnostic, using IStoreProvider to abstract storage engines like LevelDB, RocksDB, or in-memory dictionaries.
• StoreFactory maintains a global registry of providers, with MemoryStoreProvider registered by default for testing and ephemeral nodes.
• IStore and IStoreSnapshot provide the low-level API, where snapshots offer point-in-time isolation and atomic commit semantics.
• StoreCache extends DataCache to bridge high-level change tracking with low-level storage, implementing read-through caching and write-through staging to snapshots.
• Two-phase commit in StoreCache.Commit() first flushes the change set to the snapshot, then atomically persists the snapshot batch to the underlying store.
• NeoSystem exposes convenience methods like GetSnapshot() and StoreView to provide ready-to-use StoreCache instances for blockchain operations.

Frequently Asked Questions

What is the difference between StoreCache and DataCache?

DataCache is an abstract base class that maintains an in-memory change set (tracking added, updated, and deleted entries) and defines the public API for cache operations. StoreCache is the concrete implementation that connects this abstract cache to actual storage by implementing the protected internal methods (AddInternal, TryGetInternal, etc.) to call IStore or IStoreSnapshot methods. While DataCache handles the logic of what to cache, StoreCache handles how to read from and write to the underlying persistence layer.

How do I implement a custom storage provider for Neo?

To implement a custom storage provider, create a class that implements IStoreProvider with a Name property and a GetStore(string? path) method returning an IStore implementation. Your IStore must handle byte array keys and values, and support snapshots via GetSnapshot(). Register your provider at application startup using StoreFactory.RegisterProvider(new MyProvider()), after which StoreFactory.GetStore("myprovider", "path") will return your store instance.

When should I use IStoreSnapshot instead of IStore directly?

Use IStoreSnapshot when you need isolated, transactional access to storage that can be committed or discarded atomically. Snapshots are essential during block validation, where you must simulate state changes to verify transactions without affecting the global state until the block is confirmed. If the validation fails, you simply dispose the snapshot without calling Commit(). Use IStore directly only for read-only operations or when you explicitly want unbuffered, immediate writes without isolation guarantees.

How does StoreCache ensure atomic commits?

StoreCache ensures atomicity through a two-phase commit process. When you call Commit(), the method first invokes base.Commit(), which iterates through the internal change set and calls the protected AddInternal, UpdateInternal, and DeleteInternal methods. These methods write to the underlying IStoreSnapshot's batch buffer (not directly to disk). After the change set is fully staged, StoreCache calls _snapshot.Commit(), which atomically writes the entire batch to the persistent storage. If any step fails or if the cache is disposed before Commit(), the snapshot is discarded, leaving the underlying store unchanged.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}