How zvec Handles Index Persistence and Recovery: Manifest-Based Versioning in Alibaba's Vector Database

zvec implements crash-safe index persistence through atomic manifest files that capture complete index state, enabling deterministic recovery by reloading the latest manifest and reopening segments in read-only mode.

The alibaba/zvec vector database engine ensures data durability across process restarts through a sophisticated manifest-based architecture. This article examines how zvec handles index persistence and recovery by analyzing the source code implementation of atomic writes, version management, and crash recovery protocols.

The Manifest-Based Architecture

Atomic Manifest Files

Every state change in zvec culminates in a new MANIFEST_<N> file written to the collection directory. The VersionManager ensures atomicity by writing the new manifest completely before removing the previous one, guaranteeing that a crash never leaves the index in an inconsistent state. This approach is implemented in src/db/index/common/version_manager.cc at lines 88-96.

Version State Management

The Version object defined in src/db/index/common/version_manager.h encapsulates the complete index state including the schema, segment metadata list, id-map suffix, delete-snapshot suffix, next segment ID, and the enable_mmap flag. When Version::Save() is called, it serializes this state into a protobuf proto::Manifest structure that serves as the canonical on-disk representation.

Persistence Flow: From Memory to Disk

The write path follows a strict ordering to ensure durability before a new manifest is committed.

Flush All Write Buffers

First, SegmentImpl::flush() (located in src/db/index/segment/segment.cc at lines 2079-2122) flushes the write-ahead log (WAL), forward stores, and all column indexes (scalar, inverted, and vector). This guarantees that every on-disk structure contains the most recent data before any version change occurs.

Update the Version Object

After successful flushes, the in-memory Version is populated with the latest schema, segment meta list, id-map suffix, delete-snapshot suffix, next segment ID, and the enable_mmap flag.

Serialize and Write the Manifest

Version::Save() converts the Version into the protobuf proto::Manifest format. The VersionManager::flush() method writes this to a file named MANIFEST_<N> where <N> is a monotonically increasing version ID. This operation is performed in src/db/index/common/version_manager.cc.

Commit the Version

Finally, VersionManager::flush() takes a lock, increments version_id_, and deletes the old manifest file. This makes the operation crash-safe because the new file is fully written and synced before the old one is removed.

Recovery Flow: Rebuilding Index State

On startup, zvec reconstructs the entire index state by reading the latest manifest and reopening all persisted components.

Load the Latest Manifest

VersionManager::Recovery() (implemented in src/db/index/common/version_manager.cc at lines 188-199) scans the collection directory to find the highest-numbered MANIFEST_* file. It deserializes this file into a Version object via Version::Load().

Restore Collection State

CollectionImpl::recovery() (in src/db/collection.cc at lines 32-49) obtains the VersionManager, extracts the Version, and recreates the schema and enable_mmap flag from the persisted state.

Recover ID-Map and Delete Store

Using the stored id_map_path_suffix and delete_snapshot_path_suffix from the Version, CollectionImpl::recover_idmap_and_delete_store() (lines 94-106 of collection.cc) opens the on-disk id-map and delete-snapshot files.

Re-open Persisted Segments

For each SegmentMeta in Version.persisted_segment_metas(), the recovery loop (lines 56-70 of collection.cc) calls Segment::Open() with read_only = true. This reopens all segments that were persisted before the crash.

Re-open the Writing Segment

The segment that was being built when the process stopped is also reopened via Segment::Open() using v.writing_segment_meta() (lines 75-83 of collection.cc). This allows writes to continue from the next segment ID.

Restore the Segment-ID Allocator

Finally, segment_id_allocator_.store(v.next_segment_id()) (lines 84-85 of collection.cc) ensures that newly created segments receive fresh, monotonically increasing IDs without collision.

Key Design Principles

• Manifest-Based Versioning – The complete index state is captured in a single protobuf (proto::Manifest). This makes the on-disk format easy to extend and guarantees atomicity: a new file is written before the old one is removed.

• Lock-Free Reads – All readers use the Version that was loaded at startup. Because manifests are immutable once written, concurrent reads never see partially-written state.

• Separate Flush Layers – Each storage component (WAL, forward stores, RocksDB column families, id-map) implements a flush() method. The segment flush orchestrates them, and VersionManager::flush() is called only after every component succeeds.

• MMap Opt-In – The enable_mmap flag is persisted in the manifest and restored on recovery, allowing the collection to reuse memory-mapped files without extra configuration.

Summary

• zvec persists index state through atomic manifest files (MANIFEST_<N>) that capture the complete collection schema, segment metadata, and auxiliary stores.
• The persistence flow flushes all storage layers (WAL, forward stores, column indexes) before serializing a new Version to disk and atomically replacing the old manifest.
• Recovery scans for the latest manifest, deserializes the Version, and sequentially reopens the id-map, delete store, and all segments (both persisted and the active writing segment) in read-only mode.
• The design ensures crash safety through immutable manifests and lock-free reads, with separate flush layers preventing partial write corruption.

Frequently Asked Questions

What is the manifest file in zvec?

The manifest file (named MANIFEST_<N> where <N> is a monotonically increasing version ID) is a protobuf-serialized proto::Manifest that stores the complete state of a zvec collection. It contains the schema, segment metadata, id-map suffix, delete-snapshot suffix, next segment ID, and configuration flags like enable_mmap. According to the source code in src/db/index/common/version_manager.cc, a new manifest is written atomically before the old one is removed, ensuring crash safety.

How does zvec ensure crash safety during writes?

zvec achieves crash safety through a two-phase commit process. First, SegmentImpl::flush() (in src/db/index/segment/segment.cc at lines 2079-2122) ensures all write buffers, WAL entries, and column indexes are synced to disk. Only after these layers succeed does VersionManager::flush() write a new manifest file and delete the previous one. Because the new manifest is fully persisted before the old manifest is removed, a crash at any point leaves the index in a valid state—either at the old version or the new one, never in between.

What happens if the manifest file is corrupted?

If the manifest file is corrupted, VersionManager::Recovery() (implemented in src/db/index/common/version_manager.cc at lines 188-199) will fail to deserialize the protobuf into a Version object. Since the manifest is the source of truth for the entire collection state—including segment locations, schema, and metadata—corruption effectively makes the collection unrecoverable through standard means. The design assumes underlying storage reliability; for protection against storage faults, external backup or replication mechanisms would be required.

Does zvec support point-in-time recovery?

zvec maintains a linear history of manifests through monotonically increasing version IDs (MANIFEST_<N>), but the VersionManager implementation in src/db/index/common/version_manager.cc only retains the latest manifest during normal operation, deleting the previous one immediately after a successful write. Therefore, standard zvec does not support point-in-time recovery to arbitrary historical states out of the box. However, the architecture theoretically supports this capability if the system were modified to archive manifest files rather than delete them, since each manifest represents a complete, consistent snapshot of the index at a specific version.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}