How DS4 Native Agent Session Management Handles KV Cache Persistence and Resumption

The DS4 native agent embeds the KV-cache directly into session snapshots, saving a binary payload that includes both model runtime state and serialized cache contents, then rebuilds the in-memory KV-store on resumption to enable generation continuation without re‑prefilling.

DS4 (the native agent implementation by antirez) provides robust KV cache persistence and resumption through integrated session snapshot management. This mechanism allows long-running inference sessions to survive interruptions and resume with cached attention keys intact, eliminating expensive re-prefill operations. The implementation spans ds4.c, ds4_kvstore.c, and ds4_agent.c, with clear separation between session orchestration and low-level KV-store serialization.

Snapshot Creation: Saving the KV Cache

Session persistence begins when ds4_session_save_snapshot() triggers a checkpoint. This function delegates to ds4_session_save_payload() to construct the complete session payload.

In ds4_session_save_payload() at ds4.c:50094, the KV-cache serialization occurs through dedicated store functions:

// Called within ds4_session_save_payload()
ds4_kvstore_store_live_prefix_text(kvstore, writer);  // Serialize live tokens
ds4_kvstore_store_len(kvstore, writer);               // Write cache length

These functions emit:

• Live prefix tokens currently in the cache
• Cache header metadata
• Individual KV entries with their attention keys and values

The cache directory location is determined by the agent_worker structure's cache_dir field, defined at ds4_agent.c:0110.

Header Structure and Metadata Preservation

Before writing entries, ds4_kvstore_fill_header() (at ds4_kvstore.c:393) populates the fixed header structure:

Field	Purpose
Model ID	Identifies the associated model architecture
Quantization bits	Tracks compression level for cache entries
Reason code	Indicates why the snapshot was created
Token count	Current cache occupancy
Hit count	Access statistics for eviction decisions
Context size	Maximum sequence length

This header enables cross-session validation and informs the eviction policy when the cache is restored.

Session Resumption: Rebuilding the KV Cache

Loading reverses the serialization process. ds4_session_load_snapshot() calls ds4_session_load_payload(), which invokes ds4_kvstore_open() at ds4.c:50088 to reconstruct the cache:

// Reconstruction flow in ds4_kvstore_open()
ds4_kvstore_read_entry_file(kvstore, entry_path);  // Load each cached entry

The restoration process:

1. Scans the snapshot directory for entry files
2. Reads each entry's keys, values, and metadata
3. Rebuilds the in-memory hash table structure
4. Restores timestamp and hit-count fields for eviction continuity

Eviction Policy Persistence

The LRU-style eviction survives session interruptions because ds4_kvstore_evict() (at ds4_kvstore.c:561) relies on persisted entry fields:

• Timestamps — record last access time
• Hit counts — track frequency of use

When resumed, entries retain their original eviction priority, ensuring consistent cache behavior across saves and loads.

Integration with Agent Worker Lifecycle

The agent_worker structure in ds4_agent.c coordinates persistence operations:

Component	Location	Responsibility
cache_dir	ds4_agent.c:0110	Defines snapshot storage path
Snapshot trigger	ds4_agent.c	/save command or automatic checkpoint
Session attach	ds4_agent.c:0187	Binds restored session to worker thread

After ds4_kvstore_open() completes, ds4_session_create() attaches the restored state. The agent skips prefill initialization because the KV-cache already contains computed attention states.

Key Files in the Persistence Stack

Understanding the codebase organization clarifies where modifications affect KV cache persistence:

• ds4.c — Orchestrates ds4_session_save_payload() and ds4_session_load_payload(); entry points for snapshot operations
• ds4_kvstore.c / ds4_kvstore.h — Implements the on-disk format, header management, and ds4_kvstore_read_entry_file()
• ds4_agent.c — Houses agent_worker and session lifecycle management
• ds4_server.c — Exposes snapshot API for remote client requests
• tests/test_gpu_lookup_cache_strict.c, tests/test_metal_session_batch.c — Validate cache persistence across batch operations

Performance Implications

The embedded KV-cache approach provides measurable benefits for session resumption:

• Zero re-prefill latency — Attention keys are available immediately
• Bounded deserialization cost — Linear in cache size, not model depth
• Predictable memory — Snapshot size equals cache size plus fixed overhead

Trade-offs include snapshot file size (proportional to cached tokens) and serialization overhead during active generation. The implementation mitigates this through asynchronous checkpointing where possible.

Summary

• ds4_session_save_snapshot() initiates persistence by calling ds4_session_save_payload(), which serializes the KV-cache via ds4_kvstore_store_live_prefix_text() and ds4_kvstore_store_len()
• Header metadata in DS4_KVSTORE_FIXED_HEADER preserves model configuration, statistics, and eviction state through ds4_kvstore_fill_header()
• Resumption executes ds4_kvstore_open() in ds4_session_load_payload(), rebuilding entries with ds4_kvstore_read_entry_file() and restoring timestamps for consistent eviction
• Cache directory location is configured in agent_worker.cache_dir at ds4_agent.c:0110
• Generation continuation skips prefill because the restored session attaches with populated KV-cache at ds4_agent.c:0187

Frequently Asked Questions

How does DS4 determine where to save session snapshots?

The agent_worker structure contains a cache_dir field defined at ds4_agent.c:0110. This directory path stores all snapshot files and KV-cache entries for that worker instance.

What happens to the eviction policy when a session is resumed?

Eviction state persists because ds4_kvstore_read_entry_file() restores each entry's timestamp and hit count. The ds4_kvstore_evict() function at ds4_kvstore.c:561 then applies the same LRU-style selection criteria as before the save.

Can a resumed session continue generation without reprocessing previous tokens?

Yes. The key benefit of DS4's KV cache persistence is that ds4_session_create() attaches the restored cache directly, making prefill unnecessary. Attention lookups proceed immediately using the deserialized keys and values.

What triggers automatic snapshot creation in the native agent?

While manual /save commands explicitly invoke ds4_session_save_snapshot(), the agent may also checkpoint based on runtime heuristics. The reason_code field in the KV-store header (set by ds4_kvstore_fill_header()) records the trigger cause for diagnostic purposes.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}