# CCR Reversible Compression in Headroom: How the Compress-Cache-Retrieve Architecture Works

> Discover CCR reversible compression in Headroom. Learn how Compress-Cache-Retrieve perfectly reconstructs original data using BLAKE3 hashing for aggressive token compression.

- Repository: [Tejas Chopra/headroom](https://github.com/chopratejas/headroom)
- Tags: deep-dive
- Published: 2026-06-12

---

**CCR (Compress-Cache-Retrieve) is Headroom's architectural layer that enables aggressive token-level compression while preserving the ability to perfectly reconstruct original data through a BLAKE3 hash-based caching system.**

CCR reversible compression is the core mechanism that allows Headroom to aggressively reduce token counts for large tool outputs without permanent information loss. According to the chopratejas/headroom source code, this architecture stores original payloads in an LRU cache keyed by cryptographic hashes, enabling the LLM to retrieve exact data on demand through an injected `headroom_retrieve` tool.

## The Four-Phase CCR Architecture

The CCR system operates through four coordinated components that work across the request lifecycle.

### Phase 1: Compression Store and Hash Generation

When a transform like **SmartCrusher** compresses a large output (JSON arrays, log files, etc.), the system simultaneously stores the **full original payload** in an LRU cache. This happens in [`crates/headroom-core/src/ccr/mod.rs`](https://github.com/chopratejas/headroom/blob/main/crates/headroom-core/src/ccr/mod.rs), where the `compute_key` function generates a 24-character BLAKE3 hash of the content and `marker_for` produces the insertion marker `<<ccr:HASH>>` that replaces the original data in the compressed block.

```rust
use headroom_core::ccr::{compute_key, marker_for};

let payload = r#"[
    {"ts":1,"cpu":45},
    {"ts":2,"cpu":45},
    // … many more items …
]"#;

let hash = compute_key(payload.as_bytes());
let marker = marker_for(&hash);
println!("Compressed block will contain: {}", marker);
// Example output: <<ccr:1a2b3c4d5e6f7g8h9i0j1k>>

```

### Phase 2: Tool Injection

The proxy automatically injects a **`headroom_retrieve`** tool definition into the LLM's tool list. The compressed output contains the CCR marker, signaling to the model that retrieval is possible. The tool schema is defined in [`wiki/ccr.md`](https://github.com/chopratejas/headroom/blob/main/wiki/ccr.md) under *CCR Phase 3*, specifying the parameters needed to fetch cached data.

### Phase 3: Response Handler

After the LLM calls `headroom_retrieve`, the **CCR response handler** processes the request in [`ccr/response_handler.py`](https://github.com/chopratejas/headroom/blob/main/ccr/response_handler.py). It extracts the hash from the tool call, looks up the original data in the cache, and returns the full payload (or a BM25-filtered subset) to continue the conversation. This mechanics remains invisible to the end client.

### Phase 4: Context Tracker

Across multiple conversation turns, the **Context Tracker** in [`ccr/context_tracker.py`](https://github.com/chopratejas/headroom/blob/main/ccr/context_tracker.py) maintains state about which hashes were created. When a later query references previously compressed data, the system can proactively expand the relevant context without requiring explicit user intervention.

## Implementation Examples

### Using the headroom_retrieve Tool Client-Side

When integrating with Headroom's Python client, the retrieval process is handled automatically, but the tool call structure follows this pattern:

```python
from headroom import HeadroomClient, OpenAIProvider

# Wrap the original OpenAI client

base = OpenAI(api_key="...")
client = HeadroomClient(original_client=base, provider=OpenAIProvider())

# Send a request that will be compressed

resp = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Give me all logs from the last hour"}],
    headroom_mode="optimize",
)

# The LLM may respond with a tool call:

# {

#   "tool_calls": [

#     {

#       "id": "call_1",

#       "function": {

#         "name": "headroom_retrieve",

#         "arguments": "{\"hash\":\"<<ccr:abc123def456>>\"}"

#       }

#     }

#   ]

# }

# The proxy automatically handles the call, fetches the original logs from the CCR

# store, and returns the final answer to the user.

print(resp.choices[0].message.content)

```

### Triggering Proactive Context Expansion

The Context Tracker enables automatic dereferencing when follow-up questions relate to previously compressed data:

```python

# After several turns, the user asks a follow-up:

follow_up = client.chat.completions.create(
    model="gpt-4o",
    messages=[
        {"role": "assistant", "content": "Here are the top 5 error spikes."},
        {"role": "user", "content": "Show me the full logs for the biggest spike."},
    ],
    headroom_mode="optimize",
)

# The Context Tracker sees that the earlier compressed block (hash "abc123")

# likely contains the needed data, automatically injects a retrieval tool call,

# and the LLM receives the full logs without the user needing to know the hash.

print(follow_up.choices[0].message.content)

```

## Key Source Files

- **[`crates/headroom-core/src/ccr/mod.rs`](https://github.com/chopratejas/headroom/blob/main/crates/headroom-core/src/ccr/mod.rs)** – Core CCR utilities including `compute_key` for BLAKE3 hashing and `marker_for` for generating CCR markers.
- **[`wiki/ccr.md`](https://github.com/chopratejas/headroom/blob/main/wiki/ccr.md)** – Human-readable specification of CCR phases, the `headroom_retrieve` tool schema, and usage examples.
- **[`ccr/tool_injection.py`](https://github.com/chopratejas/headroom/blob/main/ccr/tool_injection.py)** – Implements the injection of the retrieval tool into the LLM's available function set.
- **[`ccr/response_handler.py`](https://github.com/chopratejas/headroom/blob/main/ccr/response_handler.py)** – Detects CCR tool calls and manages the lookup and return of cached original data.
- **[`ccr/context_tracker.py`](https://github.com/chopratejas/headroom/blob/main/ccr/context_tracker.py)** – Tracks compressed hashes across conversation turns and drives proactive expansion.
- **[`wiki/ARCHITECTURE.md`](https://github.com/chopratejas/headroom/blob/main/wiki/ARCHITECTURE.md)** – High-level documentation linking the compression store, tool injection, and response handler components.

## Summary

- **CCR (Compress-Cache-Retrieve)** enables aggressive token compression while guaranteeing perfect reconstruction of original data.
- The system uses **24-character BLAKE3 hashes** to key an LRU cache storing original payloads before compression.
- The **`headroom_retrieve`** tool allows LLMs to request specific cached data by hash when needed.
- **Context Tracker** maintains compression state across turns, enabling proactive expansion without user intervention.
- Because original data is always cached, CCR reversible compression eliminates the trade-off between token savings and information loss.

## Frequently Asked Questions

### How does CCR ensure no data loss during compression?

CCR stores the exact original payload in a cache keyed by a cryptographically strong hash before any compression occurs. The compressed block contains only a short marker (`<<ccr:HASH>>`), but the full data remains accessible via the `headroom_retrieve` tool. This guarantees that if the LLM needs the complete dataset, it can retrieve the identical original content that was cached.

### What hash algorithm does CCR use for cache keys?

Headroom uses **BLAKE3** to generate cache keys. In [`crates/headroom-core/src/ccr/mod.rs`](https://github.com/chopratejas/headroom/blob/main/crates/headroom-core/src/ccr/mod.rs), the `compute_key` function produces a 24-character hash from the payload bytes, which `marker_for` then formats into the standard `<<ccr:HASH>>` marker inserted into compressed blocks.

### How does the LLM know when to retrieve compressed data?

The proxy injects the `headroom_retrieve` tool definition into the LLM's tool list at the start of the conversation. When the model encounters a CCR marker in the compressed context or determines it needs more detail, it can invoke this tool with the specific hash to fetch the original data. The Context Tracker may also proactively trigger retrievals when it detects references to previously compressed content.

### Is CCR reversible compression compatible with all LLM providers?

Yes, CCR operates at the proxy layer rather than requiring specific provider support. As long as the LLM supports function calling or tool use, the `headroom_retrieve` tool can be injected into the available tools list. The architecture is provider-agnostic and works with any model that can generate structured tool calls, including OpenAI, Anthropic, and OpenAI-compatible APIs.