How the Native ds4-agent Manages Sessions and Persists the KV Cache

The native ds4-agent employs a dedicated worker thread to own the live ds4_session and ds4_kvstore, persisting conversational state through SHA-identified session snapshots and a content-addressable on-disk cache that survives process restarts.

The antirez/ds4 repository provides a high-performance inference front-end that maintains conversational continuity across restarts. Understanding how the native ds4-agent handles session management and key-value cache durability requires examining the dual-threaded architecture in ds4_agent.c and the persistent storage layer implemented in ds4_kvstore.c.

Dual-Threaded Architecture Overview

The ds4-agent runs as a single-process application with strict separation between interactive I/O and inference computation.

UI Thread Responsibilities

The primary thread handles terminal input/output, command parsing, and rendering. It communicates with the computational backend through thread-safe flags and condition variables rather than direct memory access.

Worker Thread Ownership

A secondary worker thread exclusively owns two critical structures:

• ds4_session *session – The live conversational context containing token history and metadata.
• ds4_kvstore – The on-disk key-value cache that stores previously generated token sequences.

All access to these structures is synchronized via a mutex (mu) and condition variable (cond) defined in the agent_worker struct, allowing the UI to safely query session_dirty and user_activity states without blocking inference.

Session Management Lifecycle

Initialization and Session Loading

When the agent starts, it allocates an agent_worker structure that contains the ds4_session *session field. The worker generates a session_sha identifier derived from the system message and initial prompt. If a previous session file exists in the cache directory, the code calls ds4_session_load_payload() to restore the token transcript; otherwise, it initializes a fresh session via ds4_session_new().

This logic resides in ds4_agent.c within the worker initialization routines (around line 1070).

Runtime Synchronization

During operation, the worker thread runs the ds4_engine and appends new tokens to the session. The UI thread queries state through synchronized flags (session_dirty, user_activity) protected by the worker's mutex, ensuring lock-free observation of inference progress.

Persistence and Shutdown

The worker transitions to AGENT_WORKER_SAVING state when the save_requested flag is set. In this state, it invokes ds4_session_save_payload() to serialize the session's token history and KV cache references to disk. On normal termination, the shutdown sequence again triggers this save routine, ensuring the latest state is flushed before the process exits.

KV Cache Persistence Mechanism

Opening the Cache Directory

The KV cache persists in a dedicated cache_dir (default ~/.cache/ds4/kv). At startup, ds4_kvstore_open() scans this directory for *.kv files and constructs an in-memory list of ds4_kvstore_entry structures. Each entry maps to a content-addressed file on disk, indexed by SHA-1 hash of the token sequence.

This implementation is found in ds4_kvstore.c (line 609).

Storing Token Sequences

New token sequences are written via ds4_kvstore_store_len(), which creates entry files named by the SHA-1 hash of the token content. Each file contains a fixed-size header storing metadata including the model ID, quantization bits, token count, and creation timestamp. This content-addressable approach ensures identical prompts across different sessions reuse the same cache file.

See ds4_kvstore.c line 943 for the storage implementation.

Eviction and Compaction

When the cache size exceeds the budget_mb parameter set during ds4_kvstore_open(), the worker enters AGENT_WORKER_COMPACTING state and calls ds4_kvstore_evict(). This function calculates an eviction score for each entry based on hit count, age, and context size, removing low-scoring files until the cache fits within the allocated budget.

The eviction logic is implemented in ds4_kvstore.c (line 561).

Integration with Session Snapshots

ds4_session_save_payload() writes both the session state and a reference to the KV cache directory. On reload, ds4_session_load_payload() reconstructs the in-memory session and reconnects it to the existing KV cache, restoring all previously computed token-to-vector mappings without regeneration.

Practical Implementation Example

/* Initialize worker and session structures */
agent_worker *w = calloc(1, sizeof(*w));
w->cache_dir = strdup("~/.cache/ds4/kv");
w->session = ds4_session_new(&engine_opts);

/* Open the persistent KV cache with 1GB budget */
ds4_kvstore_open(&w->kvstore, w->cache_dir, 1024, false, 
                 ds4_kvstore_default_options());

/* Trigger session persistence */
if (w->save_requested) {
    FILE *fp = fopen("session.snapshot", "wb");
    ds4_session_save_payload(w->session, fp, errbuf, sizeof(errbuf));
    fclose(fp);
}

/* Compact cache when budget exceeded */
if (w->compact_requested) {
    ds4_kvstore_refresh(&w->kvstore);  // Evict low-score entries
}

Summary

• The ds4-agent uses a dedicated worker thread to exclusively own the ds4_session and ds4_kvstore, preventing UI blocking while maintaining thread safety through mutex and condition variable synchronization.
• Session persistence relies on ds4_session_save_payload() and ds4_session_load_payload() using SHA-derived session identifiers to serialize token transcripts.
• The KV cache persists in ~/.cache/ds4/kv via ds4_kvstore_open(), ds4_kvstore_store_len(), and intelligent eviction through ds4_kvstore_evict() when the configured budget_mb is exceeded.
• State machine transitions including AGENT_WORKER_SAVING and AGENT_WORKER_COMPACTING ensure durable, consistent storage without interrupting inference operations.

Frequently Asked Questions

How does ds4-agent recover sessions after a process crash?

The agent relies on explicit save points rather than write-ahead logging. When ds4_session_save_payload() executes—triggered by user request or graceful shutdown—it writes a complete token transcript to disk. On restart, ds4_session_load_payload() reconstructs the conversation state from this file if present in the cache directory, restoring the exact session context.

What determines the filename for KV cache entries?

ds4_kvstore_store_len() generates filenames using the SHA-1 hash of the token sequence content. This content-addressable storage ensures that identical prompts produce identical hashes, maximizing cache reuse across different sessions and model invocations.

How does the agent prevent unlimited disk usage by the KV cache?

The ds4_kvstore_evict() function enforces the budget_mb limit specified during ds4_kvstore_open(). It ranks entries using a composite score combining access frequency, recency, and context size, then removes the lowest-ranked files until the total cache size falls within the allocated budget.

Which source files implement the core persistence logic?

Session management resides in ds4_agent.c (worker thread orchestration) and ds4.c (serialization routines ds4_session_save_payload/ds4_session_load_payload). The KV cache implementation lives entirely in ds4_kvstore.c, handling file I/O, content addressing, and eviction policies.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}