How to Save and Restore Agent Sessions Using KV Cache Files in ds4 Session persistence

This guide explains how to use ds4's on-disk key-value cache to checkpoint agent execution state and resume sessions without reprocessing entire prompts.

The antirez/ds4 repository implements a durable KV cache system that enables efficient session persistence for LLM agents. By storing token prefixes and their corresponding graph states to disk, agents can restore previous contexts instantly—eliminating redundant computation and enabling long-running conversational workflows.

Initializing the KV Store

Before saving or loading sessions, you must open a KV store that manages the checkpoint directory. The ds4_kvstore_open() function in ds4_kvstore.c (lines 1062–1085) creates or opens the cache directory, configures storage budgets, and sets compatibility policies.

The function signature accepts:

• A directory path for checkpoint files
• A budget limit in megabytes
• A flag to reject checkpoints with mismatched quantization
• Logging callbacks for operational visibility

#include "ds4_kvstore.h"

ds4_kvstore kv;
ds4_kvstore_options opt = ds4_kvstore_default_options();

bool ok = ds4_kvstore_open(
    &kv,
    "./kvcache",        /* checkpoint directory */
    500,                /* 500 MiB budget */
    false,              /* allow different quantization */
    opt,
    "my-agent",         /* logger identifier */
    my_log_callback,    /* optional log function */
    NULL);              /* user data for logger */

if (!ok) {
    fprintf(stderr, "Failed to initialise KV cache\n");
    exit(1);
}

After initialization, the store is ready to accept checkpoints. Close the store with ds4_kvstore_close() (lines 1446–1450) when your application shuts down to release resources.

Saving Agent Sessions to KV Cache Files

To persist an agent session, use ds4_kvstore_store_live_prefix() or ds4_kvstore_store_live_prefix_text() found in ds4_kvstore.c (lines 1566–1655). These functions serialize the current token prefix and engine state into an atomic, self-describing file.

The storage process follows these steps:

1. Render tokens to plain text — converts the token ID sequence to human-readable bytes
2. Compute SHA-1 fingerprint — hashes the rendered text via ds4_kvstore_sha1_bytes_hex() (lines 221–227)
3. Serialize engine payload — captures the graph state to a temporary file
4. Write structured header — encodes metadata (magic KVC, version, quantization, reason code, timestamps)
5. Atomic rename — commits the file as <sha>.kv only after successful writes

char errbuf[256];
bool stored = ds4_kvstore_store_live_prefix(
    &kv,
    engine,               /* ds4_engine* */
    sess,                 /* ds4_session* */
    tokens,               /* token prefix to checkpoint */
    ds4_kvstore_store_len(&kv, tokens->len),
    "cold",               /* reason: DS4_KVSTORE_REASON_COLD */
    NULL,                 /* no trailer hooks */
    errbuf,
    sizeof(errbuf));

if (!stored) {
    fprintf(stderr, "KV store failed: %s\n", errbuf);
}

The fixed 56-byte header (DS4_KVSTORE_FIXED_HEADER) contains critical metadata created by ds4_kvstore_fill_header() (lines 393–415):

• Magic identifier and version
• Quantization mode (2-bit or 4-bit)
• Model ID and token count
• Hit counters and context size
• Payload ABI identifier and timestamps

Before writing, ds4_kvstore_file_size_fits() validates that the new checkpoint respects the configured budget. If exceeded, ds4_kvstore_evict() (lines 616–667) removes low-value checkpoints using an eviction score that prioritizes recent, frequently-accessed entries.

Restoring Agent Sessions from KV Cache Files

To restore a saved session, call ds4_kvstore_try_load_text() from ds4_kvstore.c (lines 1242–1315). This function searches for checkpoints matching the byte prefix of an incoming prompt and validates integrity before deserialization.

The restoration process:

1. Locate candidate files — scans for <sha>.kv files in the cache directory
2. Verify header integrity — checks magic, version, and recomputes SHA-1 of stored text
3. Validate prefix match — confirms stored text is a byte prefix of the new prompt
4. Deserialize payload — restores engine graph state into the session
5. Return recovery metadata — reports tokens recovered and optional load results

int loaded = ds4_kvstore_try_load_text(
    &kv,
    engine,
    sess,
    prompt_text,          /* new prompt to match */
    NULL,                 /* optional: effective prompt output */
    NULL,                 /* optional: detailed load result */
    NULL,                 /* no trailer hooks */
    false);               /* standard protocol (not responses) */

if (loaded > 0) {
    printf("Restored %d tokens from KV cache!\n", loaded);
} else {
    printf("No matching checkpoint – normal processing\n");
}

A positive return value indicates successful token recovery. The agent can immediately resume generation from the cached state rather than recomputing attention from scratch.

Extending Checkpoints with Trailer Hooks

For advanced use cases, ds4_kvagent session saving and restoring supports optional trailer hooks via the ds4_kvstore_trailer_hooks struct. These allow attachment of custom metadata—such as tool maps or conversation state—beyond the engine payload.

The trailer system, implemented in kv_trailer_write() (lines 778–782), serializes hook data after the payload and sets a header flag so loaders know to invoke the corresponding read hook during restoration.

Complete Workflow Example

#include "ds4_kvstore.h"

int main(void) {
    ds4_kvstore kv;
    ds4_kvstore_open(&kv, "./kvcache", 1000, false,
                     ds4_kvstore_default_options(),
                     "agent", NULL, NULL);

    /* After generating a stable prefix... */
    ds4_kvstore_store_live_prefix(&kv, engine, sess, tokens,
                                  ds4_kvstore_store_len(&kv, tokens->len),
                                  "continued", NULL, NULL, 0);

    /* Later, on new prompt arrival... */
    int recovered = ds4_kvstore_try_load_text(&kv, engine, sess,
                                              new_prompt, NULL, NULL, NULL, false);
    
    ds4_kvstore_close(&kv);
    return 0;
}

Key Source Files for KV Cache Operations

File	Purpose	Critical Functions
ds4_kvstore.c	Core persistence implementation	ds4_kvstore_open(), ds4_kvstore_store_live_prefix(), ds4_kvstore_try_load_text(), ds4_kvstore_evict()
ds4_kvstore.h	Public API declarations	Struct definitions, option defaults, hook interfaces
ds4_agent.c	High-level agent integration	Orchestrates when to checkpoint and restore
ds4.c	CLI entry point	Parses --kvcache-dir and wires KV store lifecycle

Summary

• Initialize with ds4_kvstore_open() to create or open a checkpoint directory with configurable budget and policies
• Save sessions using ds4_kvstore_store_live_prefix() to atomically write <sha>.kv files containing headers, rendered text, and engine payloads
• Restore sessions via ds4_kvstore_try_load_text() to match prompt prefixes and deserialize valid checkpoints
• Manage resources through automatic budget enforcement and usage-based eviction in ds4_kvstore_evict()
• Extend functionality with trailer hooks for custom metadata serialization beyond core engine state

Frequently Asked Questions

What filename format does ds4 use for KV cache files?

ds4 names checkpoint files <sha>.kv where <sha> is the 40-character hexadecimal SHA-1 hash of the rendered token text. This naming convention enables O(1) lookup by content fingerprint and provides intrinsic integrity verification during loading.

How does ds4 handle storage budget limits?

Before writing any checkpoint, ds4 calls ds4_kvstore_file_size_fits() to estimate whether the new file would exceed the configured megabyte budget. If so, ds4_kvstore_evict() selects victims using a scoring function that weights recency, hit frequency, and checkpoint type—preferring to remove older, less-accessed entries while preserving valuable hot checkpoints.

Can I use KV cache files across different quantization modes?

By default, ds4 allows loading checkpoints regardless of quantization. However, passing true to the reject_different_quant parameter in ds4_kvstore_open() enforces strict compatibility—attempting to load a 2-bit checkpoint into a 4-bit engine (or vice versa) will fail with an integrity error, preventing subtle numerical mismatches.

What is the difference between ds4_kvstore_store_live_prefix() and ds4_kvstore_store_live_prefix_text()?

ds4_kvstore_store_live_prefix() accepts a ds4_tokens struct and handles rendering internally, while ds4_kvstore_store_live_prefix_text() allows callers to supply pre-rendered text directly. The text variant offers flexibility when the caller has already computed string representations, potentially avoiding duplicate rendering work in specialized agent workflows.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}