# How DS4 Native Agent Session Management Handles KV Cache Persistence and Resumption

> Discover how DS4 native agent session management persists and resumes KV cache. Learn how DS4 saves and rebuilds the KV cache for seamless generation continuation.

- Repository: [Salvatore Sanfilippo/ds4](https://github.com/antirez/ds4)
- Tags: internals
- Published: 2026-08-04

---

**The DS4 native agent embeds the KV-cache directly into session snapshots, saving a binary payload that includes both model runtime state and serialized cache contents, then rebuilds the in-memory KV-store on resumption to enable generation continuation without re‑prefilling.**

DS4 (the native agent implementation by antirez) provides robust **KV cache persistence and resumption** through integrated session snapshot management. This mechanism allows long-running inference sessions to survive interruptions and resume with cached attention keys intact, eliminating expensive re-prefill operations. The implementation spans [`ds4.c`](https://github.com/antirez/ds4/blob/main/ds4.c), [`ds4_kvstore.c`](https://github.com/antirez/ds4/blob/main/ds4_kvstore.c), and [`ds4_agent.c`](https://github.com/antirez/ds4/blob/main/ds4_agent.c), with clear separation between session orchestration and low-level KV-store serialization.

## Snapshot Creation: Saving the KV Cache

Session persistence begins when `ds4_session_save_snapshot()` triggers a checkpoint. This function delegates to `ds4_session_save_payload()` to construct the complete session payload.

In `ds4_session_save_payload()` at `ds4.c:50094`, the KV-cache serialization occurs through dedicated store functions:

```c
// Called within ds4_session_save_payload()
ds4_kvstore_store_live_prefix_text(kvstore, writer);  // Serialize live tokens
ds4_kvstore_store_len(kvstore, writer);               // Write cache length

```

These functions emit:
- Live prefix tokens currently in the cache
- Cache header metadata
- Individual KV entries with their attention keys and values

The **cache directory** location is determined by the `agent_worker` structure's `cache_dir` field, defined at `ds4_agent.c:0110`.

## Header Structure and Metadata Preservation

Before writing entries, `ds4_kvstore_fill_header()` (at `ds4_kvstore.c:393`) populates the fixed header structure:

| Field | Purpose |
|-------|---------|
| Model ID | Identifies the associated model architecture |
| Quantization bits | Tracks compression level for cache entries |
| Reason code | Indicates why the snapshot was created |
| Token count | Current cache occupancy |
| Hit count | Access statistics for eviction decisions |
| Context size | Maximum sequence length |

This header enables cross-session validation and informs the **eviction policy** when the cache is restored.

## Session Resumption: Rebuilding the KV Cache

Loading reverses the serialization process. `ds4_session_load_snapshot()` calls `ds4_session_load_payload()`, which invokes `ds4_kvstore_open()` at `ds4.c:50088` to reconstruct the cache:

```c
// Reconstruction flow in ds4_kvstore_open()
ds4_kvstore_read_entry_file(kvstore, entry_path);  // Load each cached entry

```

The restoration process:
1. Scans the snapshot directory for entry files
2. Reads each entry's keys, values, and metadata
3. Rebuilds the in-memory hash table structure
4. Restores timestamp and hit-count fields for eviction continuity

## Eviction Policy Persistence

The **LRU-style eviction** survives session interruptions because `ds4_kvstore_evict()` (at `ds4_kvstore.c:561`) relies on persisted entry fields:

- **Timestamps** — record last access time
- **Hit counts** — track frequency of use

When resumed, entries retain their original eviction priority, ensuring consistent cache behavior across saves and loads.

## Integration with Agent Worker Lifecycle

The `agent_worker` structure in [`ds4_agent.c`](https://github.com/antirez/ds4/blob/main/ds4_agent.c) coordinates persistence operations:

| Component | Location | Responsibility |
|-----------|----------|----------------|
| `cache_dir` | `ds4_agent.c:0110` | Defines snapshot storage path |
| Snapshot trigger | [`ds4_agent.c`](https://github.com/antirez/ds4/blob/main/ds4_agent.c) | `/save` command or automatic checkpoint |
| Session attach | `ds4_agent.c:0187` | Binds restored session to worker thread |

After `ds4_kvstore_open()` completes, `ds4_session_create()` attaches the restored state. The agent skips prefill initialization because the KV-cache already contains computed attention states.

## Key Files in the Persistence Stack

Understanding the codebase organization clarifies where modifications affect **KV cache persistence**:

- **[`ds4.c`](https://github.com/antirez/ds4/blob/main/ds4.c)** — Orchestrates `ds4_session_save_payload()` and `ds4_session_load_payload()`; entry points for snapshot operations
- **[`ds4_kvstore.c`](https://github.com/antirez/ds4/blob/main/ds4_kvstore.c)** / **[`ds4_kvstore.h`](https://github.com/antirez/ds4/blob/main/ds4_kvstore.h)** — Implements the on-disk format, header management, and `ds4_kvstore_read_entry_file()`
- **[`ds4_agent.c`](https://github.com/antirez/ds4/blob/main/ds4_agent.c)** — Houses `agent_worker` and session lifecycle management
- **[`ds4_server.c`](https://github.com/antirez/ds4/blob/main/ds4_server.c)** — Exposes snapshot API for remote client requests
- **[`tests/test_gpu_lookup_cache_strict.c`](https://github.com/antirez/ds4/blob/main/tests/test_gpu_lookup_cache_strict.c)**, **[`tests/test_metal_session_batch.c`](https://github.com/antirez/ds4/blob/main/tests/test_metal_session_batch.c)** — Validate cache persistence across batch operations

## Performance Implications

The embedded KV-cache approach provides measurable benefits for **session resumption**:

- **Zero re-prefill latency** — Attention keys are available immediately
- **Bounded deserialization cost** — Linear in cache size, not model depth
- **Predictable memory** — Snapshot size equals cache size plus fixed overhead

Trade-offs include snapshot file size (proportional to cached tokens) and serialization overhead during active generation. The implementation mitigates this through asynchronous checkpointing where possible.

## Summary

- **`ds4_session_save_snapshot()`** initiates persistence by calling `ds4_session_save_payload()`, which serializes the KV-cache via `ds4_kvstore_store_live_prefix_text()` and `ds4_kvstore_store_len()`
- **Header metadata** in `DS4_KVSTORE_FIXED_HEADER` preserves model configuration, statistics, and eviction state through `ds4_kvstore_fill_header()`
- **Resumption** executes `ds4_kvstore_open()` in `ds4_session_load_payload()`, rebuilding entries with `ds4_kvstore_read_entry_file()` and restoring timestamps for consistent eviction
- **Cache directory** location is configured in `agent_worker.cache_dir` at `ds4_agent.c:0110`
- **Generation continuation** skips prefill because the restored session attaches with populated KV-cache at `ds4_agent.c:0187`

## Frequently Asked Questions

### How does DS4 determine where to save session snapshots?

The `agent_worker` structure contains a `cache_dir` field defined at `ds4_agent.c:0110`. This directory path stores all snapshot files and KV-cache entries for that worker instance.

### What happens to the eviction policy when a session is resumed?

Eviction state persists because `ds4_kvstore_read_entry_file()` restores each entry's timestamp and hit count. The `ds4_kvstore_evict()` function at `ds4_kvstore.c:561` then applies the same LRU-style selection criteria as before the save.

### Can a resumed session continue generation without reprocessing previous tokens?

Yes. The key benefit of DS4's **KV cache persistence** is that `ds4_session_create()` attaches the restored cache directly, making prefill unnecessary. Attention lookups proceed immediately using the deserialized keys and values.

### What triggers automatic snapshot creation in the native agent?

While manual `/save` commands explicitly invoke `ds4_session_save_snapshot()`, the agent may also checkpoint based on runtime heuristics. The `reason_code` field in the KV-store header (set by `ds4_kvstore_fill_header()`) records the trigger cause for diagnostic purposes.