How to Configure --kv-disk-dir for Session Persistence with KV Disk Caching in DS4

To enable session persistence in DS4, launch the server with --kv-disk-dir /path/to/dir alongside --kv-disk-space-mb to automatically cache KV checkpoints to disk, allowing instant session restoration after restarts without re-prefilling prompts.

The antirez/ds4 inference server minimizes latency for multi-turn conversations by persisting key-value (KV) cache states to disk. Configuring the --kv-disk-dir parameter activates automatic checkpointing of hidden attention states, eliminating the computational cost of re-processing conversation history after server restarts or session switches.

How KV Disk Caching Works in DS4

DS4 maintains the KV state of active chat sessions in RAM during operation. When --kv-disk-dir is specified, the server also serializes checkpoint files to the designated directory. Each checkpoint contains the rendered-text prefix, hidden KV payload, and visible tool-call data (e.g., response text), indexed by the SHA-1 hash of the rendered byte-prefix.

When a new request arrives for a previously cached conversation, DS4 computes the SHA-1 of the rendered byte-prefix. If a matching <sha1>.kv file exists, the server loads the checkpoint, replays the hidden KV state, and tokenizes only the new suffix. This design makes session switches and server restarts virtually free of re-prefill cost.

Configuring the KV Disk Directory

Setting up persistent storage requires pointing the server to a writable filesystem path and defining a storage budget.

Starting the Server with KV Disk Caching

Pass the --kv-disk-dir flag followed by a directory path. Combine this with --kv-disk-space-mb to set the maximum cache size in megabytes.

./ds4-server --ctx 100000 \
             --kv-disk-dir /tmp/ds4-kv \
             --kv-disk-space-mb 8192

The flag is parsed in ds4_server.c at lines 13164-13166, where the directory path is stored in the server configuration:

if (!strcmp(arg, "--kv-disk-dir")) {
    c.kv_disk_dir = need_arg(&i, argc, argv, arg);   // ds4_server.c#L13164-L13166
}

During initialization, if cfg.kv_disk_dir is set, the server invokes kv_cache_open to prepare the on-disk storage (lines 13420-13422):

if (cfg.kv_disk_dir) {
    kv_cache_open(&s.kv, cfg.kv_disk_dir, cfg.kv_disk_space_mb,
                  cfg.kv_cache_reject_different_quant, cfg.kv_cache);
}                                                   // ds4_server.c#L13420-L13422

Checkpoint Persistence on Shutdown

During graceful shutdown (e.g., via SIGTERM), DS4 iterates over all active conversation slots and evaluates whether each session contains sufficient tokens to merit caching. Valid sessions trigger kv_cache_store_current to flush the checkpoint to disk, as implemented at lines 13556-13564 in ds4_server.c:

/* Simplified flow from ds4_server.c shutdown sequence */
kv_cache_store_current(&s->kv, slot_id, token_count);

After the process exits, the checkpoint files remain in --kv-disk-dir for subsequent restarts.

Managing Disk Space and Eviction

The disk cache respects the size limit specified by --kv-disk-space-mb. When the total size of .kv files in the directory exceeds this budget, DS4 automatically evicts older checkpoints. You can tune additional parameters such as minimum token thresholds and cold-save settings, but the essential persistence mechanism requires only the directory path and space limit.

Summary

• Configure --kv-disk-dir with a writable directory path to enable automatic KV checkpoint serialization in DS4.
• Checkpoints are stored as <sha1>.kv files containing rendered prefixes, hidden attention states, and tool-call data.
• Graceful shutdown triggers kv_cache_store_current in ds4_server.c to persist active sessions that meet token thresholds.
• Startup restoration uses kv_cache_open to reload caches, eliminating re-prefill costs for existing conversations.
• Disk usage is constrained by --kv-disk-space-mb, with automatic eviction when the budget is exceeded.

Frequently Asked Questions

What file format does DS4 use for KV checkpoints?

DS4 stores checkpoints as binary files named <sha1>.kv, where the SHA-1 hash corresponds to the rendered byte-prefix of the conversation. These files contain the hidden KV payload, visible tool-call data, and metadata required to restore the exact attention state.

How does DS4 determine which sessions to cache to disk?

During shutdown, the server evaluates each active slot's token count against configurable thresholds. Sessions meeting the criteria trigger kv_cache_store_current to write the checkpoint, ensuring only sufficiently large contexts consume disk space.

Can I migrate the KV cache directory to a different server?

Yes, the checkpoint files are portable between compatible DS4 builds. Copy the contents of your --kv-disk-dir directory to the new host and launch the server with the same path. Ensure the --kv-disk-space-mb limit accommodates the transferred data.

What happens if the KV disk cache exceeds the configured space budget?

When the total size of files in --kv-disk-dir surpasses --kv-disk-space-mb, DS4 automatically evicts older checkpoints to maintain the budget. The eviction strategy prioritizes recent or frequently accessed sessions according to the internal cache management logic.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}