# How to Handle KV Cache Mismatches and Session Recovery in ds4

> Learn how ds4 handles KV cache mismatches and session recovery. Discover how ds4 invalidates sessions, removes corrupt files, and ensures inference integrity with automatic pre-fill.

- Repository: [Salvatore Sanfilippo/ds4](https://github.com/antirez/ds4)
- Tags: how-to-guide
- Published: 2026-08-08

---

**When ds4 detects a KV cache mismatch during checkpoint loading, it invalidates the live session via `ds4_session_invalidate()`, removes the corrupt cache file, and falls back to a fresh pre-fill to ensure inference integrity.**

The ds4 inference engine persists attention states and token histories in a disk-based KV cache to enable fast session resumption. When incoming prompts diverge from cached checkpoints or stored headers become corrupted, understanding how to handle KV cache mismatches and session recovery is critical for maintaining reliable production deployments. The system implements a strict verification pipeline and centralized invalidation mechanism to prevent corrupted states from propagating into live inference.

## How ds4 Detects KV Cache Mismatches

The engine validates cached checkpoints through a four-stage verification pipeline implemented in [`ds4_kvstore.c`](https://github.com/antirez/ds4/blob/main/ds4_kvstore.c). Any failure during these checks triggers immediate session invalidation and cache eviction.

### Header Validation

The verification process begins in `ds4_kvstore_read_header()`, which checks magic bytes (`KV_CACHE_MAGIC0-2`), the cache version (`KV_CACHE_VERSION`), and payload ABI compatibility (`KV_CACHE_PAYLOAD_ABI`) at lines 20-24. If the header structure does not match the current binary's expectations, the load aborts before reading any KV data, preventing ABI mismatches from corrupting the session.

### Text-Prefix Hash Verification

After header validation, ds4 computes a SHA-1 hash of the stored text prefix using `ds4_kvstore_sha1_bytes_hex()` and compares it against the hash derived from the beginning of the incoming prompt (lines 12622-12632). A mismatch indicates the cached content does not correspond to the current request, forcing the engine to treat the checkpoint as stale.

### Byte-Prefix Matching

Even with matching hashes, ds4 performs a byte-level prefix check via `ds4_kvstore_byte_prefix_match()` (lines 6670-6672) to ensure the cached text is a literal prefix of the incoming prompt. If the prompt diverges earlier than the cached sequence, the checkpoint is rejected and the session is invalidated.

### Payload Deserialization Checks

Finally, `ds4_session_load_payload()` streams the raw KV ring and model-specific state from disk, verifying that the bytes read exactly match the payload size recorded in the header (lines 12484-12512). Discrepancies here indicate file truncation or corruption, triggering immediate invalidation and file removal.

## Session Invalidation Mechanisms

When validation fails, ds4 triggers `ds4_session_invalidate()` to clear internal state flags such as `checkpoint_valid` (lines 65508-65520 in [`ds4.c`](https://github.com/antirez/ds4/blob/main/ds4.c)). This function serves as the central safety mechanism across three primary code paths.

### KV-Cache Loading Failures

In `ds4_kvstore_try_load_text()` (lines 12415-12435), failures during header reading, hash comparison, or prefix matching immediately call `ds4_session_invalidate(s->session)` followed by `unlink(path)` to delete the corrupt file from the cache directory.

### Session Rewrite Failures

During request processing in [`ds4_server.c`](https://github.com/antirez/ds4/blob/main/ds4_server.c) (lines 3885-3894), if `ds4_session_rewrite_from_common()` returns `DS4_SESSION_REWRITE_REBUILD_NEEDED`, the server discards the live KV state and attempts to load an older checkpoint. If none are usable, the session is invalidated and the engine performs a full pre-fill.

### Explicit User Actions

Both [`ds4_cli.c`](https://github.com/antirez/ds4/blob/main/ds4_cli.c) (line 1372) and [`ds4_agent.c`](https://github.com/antirez/ds4/blob/main/ds4_agent.c) (line 4273) invoke `ds4_session_invalidate()` when users clear chat history or when decoding steps abort due to runtime errors. This ensures stale KV states do not persist across user operations or error conditions.

## Implementing Session Recovery

The recovery flow follows a strict hierarchy: invalidate the current session, attempt to locate a newer compatible checkpoint, and fall back to full pre-fill if necessary. The following examples demonstrate loading cached checkpoints and handling mismatches programmatically.

```c
/* --------------------------------------------------------------
   Example: Load a cached KV checkpoint and recover from a mismatch.
   -------------------------------------------------------------- */
#include "ds4.h"
#include "ds4_kvstore.h"

void run_prompt(ds4_engine *engine, const char *prompt) {
    ds4_session *session = NULL;
    ds4_session_create(&session, engine, 4096);   // allocate a session

    // Try to recover from KV cache
    ds4_tokens effective = {0};
    ds4_kvstore_load_result result = {0};
    int loaded = ds4_kvstore_try_load_text(
        &engine->kvstore,          // KV store singleton
        engine,                    // engine for tokenization
        session,                   // live session to populate
        prompt,                    // incoming prompt text
        &effective,                // will contain the prompt tokens
        &result,                   // optional diagnostics
        NULL,                      // no extra trailer hooks
        false);                    // no response-protocol flag

    if (loaded > 0) {
        /* Cache hit – resume from the stored state. */
        printf("Cache hit: resumed at %d tokens\n", loaded);
        ds4_session_sync(session, &effective, NULL, 0);
    } else {
        /* Cache miss or mismatch – fallback to full prefill. */
        ds4_tokens full = {0};
        ds4_tokenize_rendered_chat(engine, prompt, &full);
        ds4_session_sync(session, &full, NULL, 0);
    }

    /* ... continue generation ... */

    ds4_session_free(session);
}

```

```c
/* --------------------------------------------------------------
   Example: Explicitly invalidate a session after a runtime error.
   -------------------------------------------------------------- */
void decode_loop(ds4_session *s) {
    int token = ds4_session_argmax(s);
    if (token < 0) {
        /* Something went wrong – discard the corrupted KV state. */
        ds4_session_invalidate(s);
        return;
    }
    /* Normal processing ... */
}

```

## Key Source Files and Functions

| File | Purpose | Important Symbols |
|------|---------|-------------------|
| [`ds4_kvstore.c`](https://github.com/antirez/ds4/blob/main/ds4_kvstore.c) | Core implementation of on-disk KV cache, loading, verification, and eviction | `ds4_kvstore_try_load_text`, `ds4_kvstore_read_header`, `ds4_kvstore_byte_prefix_match`, `ds4_kvstore_sha1_bytes_hex` |
| [`ds4.c`](https://github.com/antirez/ds4/blob/main/ds4.c) | Session lifecycle, invalidation logic, and low-level graph handling | `ds4_session_invalidate`, `ds4_session_sync`, `ds4_session_load_payload` |
| [`ds4.h`](https://github.com/antirez/ds4/blob/main/ds4.h) | Public API declarations for session management and KV cache operations | `ds4_session_invalidate`, `ds4_session_rewrite_from_common`, `ds4_session_sync` |
| [`ds4_server.c`](https://github.com/antirez/ds4/blob/main/ds4_server.c) | Server-side request handling and cache integration | Cache loading error paths, rewrite failure handling |
| [`ds4_cli.c`](https://github.com/antirez/ds4/blob/main/ds4_cli.c) / [`ds4_agent.c`](https://github.com/antirez/ds4/blob/main/ds4_agent.c) | User-facing CLI and agent interfaces | Calls to `ds4_session_invalidate` after history truncation or decode failures |

## Summary

- **Four-stage verification** (header, SHA-1 hash, byte-prefix match, payload size) ensures cache integrity before resuming sessions.
- **`ds4_session_invalidate()`** serves as the central safety mechanism, clearing `checkpoint_valid` flags and forcing clean-state recovery.
- **Automatic cleanup** removes corrupt cache files via `unlink(path)` to prevent future false hits.
- **Hierarchical recovery** attempts newer checkpoints after invalidation, falling back to full pre-fill only when necessary.
- **Explicit invalidation** in CLI and agent code ensures user actions and runtime errors do not leave stale KV states.

## Frequently Asked Questions

### What triggers a KV cache mismatch in ds4?

A KV cache mismatch occurs when `ds4_kvstore_try_load_text()` detects header corruption, SHA-1 hash mismatches between stored and incoming text, failed byte-prefix comparisons, or payload size discrepancies during deserialization. Any of these validation failures trigger `ds4_session_invalidate()` and removal of the corrupt file.

### How does ds4 recover from a corrupted KV checkpoint?

The engine first invalidates the live session to clear internal state flags, then attempts to load an older compatible checkpoint. If no valid checkpoints exist, ds4 falls back to a full pre-fill of the prompt through `ds4_session_sync()`, rebuilding the KV state from scratch at the cost of additional computation.

### What is the role of `ds4_session_invalidate()`?

`ds4_session_invalidate()` is the central invalidation routine defined in [`ds4.c`](https://github.com/antirez/ds4/blob/main/ds4.c) (lines 65508-65520) that marks a session as invalid by clearing flags like `checkpoint_valid`. This ensures subsequent calls to `ds4_session_sync()` recognize the need for a clean-state rebuild rather than attempting to reuse corrupted KV memory.

### Where is the cache verification logic implemented?

The core verification pipeline resides in [`ds4_kvstore.c`](https://github.com/antirez/ds4/blob/main/ds4_kvstore.c), including header validation (`ds4_kvstore_read_header`), cryptographic hash checks (`ds4_kvstore_sha1_bytes_hex`), prefix matching (`ds4_kvstore_byte_prefix_match`), and payload loading (`ds4_session_load_payload`). Session invalidation triggers are distributed across [`ds4.c`](https://github.com/antirez/ds4/blob/main/ds4.c), [`ds4_server.c`](https://github.com/antirez/ds4/blob/main/ds4_server.c), [`ds4_cli.c`](https://github.com/antirez/ds4/blob/main/ds4_cli.c), and [`ds4_agent.c`](https://github.com/antirez/ds4/blob/main/ds4_agent.c).