# What Is CCR Reversible Compression and How Does headroom_retrieve Work?

> Learn about CCR reversible compression and how headroom_retrieve expands LLM content on demand. Understand this efficient caching and retrieval mechanism.

- Repository: [Tejas Chopra/headroom](https://github.com/chopratejas/headroom)
- Tags: deep-dive
- Published: 2026-06-18

---

**CCR (Compress-Cache-Retrieve) is a reversible compression mechanism that stores original content in a local cache keyed by a deterministic hash, replacing it with a lightweight placeholder that the LLM can expand on demand using the `headroom_retrieve` tool.**

CCR reversible compression is the core innovation in the Headroom framework that enables aggressive token reduction while guaranteeing zero information loss. When transformers like SmartCrusher or IntelligentContext drop content to fit within model context windows, the system preserves the original data in a local CCR store and substitutes it with a compact marker containing a CCR key. This architecture allows the LLM to recover the full payload later through the `headroom_retrieve` tool, as implemented in the `chopratejas/headroom` repository.

## Understanding CCR Reversible Compression

The CCR mechanism operates on a simple principle: **compress aggressively, but never delete**. When a transformer determines that a piece of content can be removed to stay within the context window, Headroom performs two atomic operations.

First, the full-size payload is written to an in-memory CCR cache and keyed by a deterministic hash (the **CCR key**). Second, the dropped content is replaced by a small placeholder containing this key. The marker informs the LLM that the original data exists and can be fetched on demand.

This design yields **90%+ token savings** from aggressive compression while ensuring zero loss—any piece the model later needs can be recovered instantly without extra HTTP traffic from the client.

### The CCR Workflow

The complete lifecycle of a compressed payload follows five distinct phases:

- **Compression**: Original content is moved to the CCR store and replaced with a placeholder containing a `ccr_key` (e.g., `{{CCR:ccr_5f4a2b}}`).
- **Detection**: The `CCRResponseHandler` class in [`headroom/ccr/response_handler.py`](https://github.com/chopratejas/headroom/blob/main/headroom/ccr/response_handler.py) monitors the LLM output stream for `headroom_retrieve` tool use blocks.
- **Retrieval**: Upon detecting a retrieval request, the handler queries the CCR store using the provided hash (and optional BM25-style query).
- **Return**: The original payload (or a filtered subset) is returned as a `functionResponse` for the LLM to continue reasoning.
- **Feedback**: If the LLM accesses stored content, the system records that the dropped content was "important," feeding this signal back into the compression policy to improve future decisions.

## How headroom_retrieve Works

The `headroom_retrieve` tool is automatically injected into the LLM's toolset according to the schema defined in [`headroom/tools.json`](https://github.com/chopratejas/headroom/blob/main/headroom/tools.json). This tool serves as the bridge between the compressed prompt and the full original content.

### Tool Schema and Invocation

The tool accepts two parameters: a required `hash` (the CCR key) and an optional `query` for BM25-style searching within the stored payload.

```json
{
  "name": "headroom_retrieve",
  "description": "Retrieve the original uncompressed content that was stored in CCR.",
  "parameters": {
    "type": "object",
    "properties": {
      "hash": { "type": "string", "description": "CCR key of the stored content" },
      "query": { "type": "string", "description": "Optional search term for BM25-style lookup", "default": "" }
    },
    "required": ["hash"]
  }
}

```

When the LLM emits a tool use block such as:

```json
{
  "type": "tool_use",
  "id": "toolu_1",
  "name": "headroom_retrieve",
  "input": { "hash": "ccr_abc123", "query": "error handling" }
}

```

The **CCRResponseHandler** intercepts this call, fetches the data from the CCR store, and returns a response like:

```json
{
  "type": "tool_response",
  "tool_use_id": "toolu_1",
  "content": { "type": "object", "content": "...original text..." }
}

```

### Implementation Architecture

The compression summary logic in [`headroom/transforms/compression_summary.py`](https://github.com/chopratejas/headroom/blob/main/headroom/transforms/compression_summary.py) generates the markers that tell the LLM a CCR key is available. The response handler then manages the round-trip automatically, ensuring the client sees no additional HTTP traffic. The proxy handles the entire operation transparently, re-injecting the enriched content back into the conversation flow.

## Code Examples

### Enabling CCR in a Session

```python
from headroom import HeadroomClient

client = HeadroomClient()

# Enable CCR (default is True)

client.configure(ccr_enabled=True)

# Run a prompt – the proxy will compress aggressively and store originals.

response = client.chat("Summarize the following log file:", files=["/path/to/big.log"])

```

The response will contain a compression summary and a CCR marker like `{{CCR:ccr_5f4a2b}}`.

### Manual Retrieval for Debugging

While the proxy resolves `headroom_retrieve` calls automatically, you can invoke the tool manually:

```python

# Manual retrieval – useful for debugging or external tools

original = client.headroom_retrieve(hash="ccr_5f4a2b")
print(original)   # prints the full, uncompressed log content

```

### Query-Based Retrieval

CCR supports BM25-style searching inside the stored payload:

```python
hits = client.headroom_retrieve(hash="ccr_5f4a2b", query="authentication failure")
print(hits)   # returns only the matching lines/sections

```

### End-to-End Roundtrip Test

This pattern mirrors the test suite in [`tests/test_transforms/test_smart_crusher_ccr_roundtrip.py`](https://github.com/chopratejas/headroom/blob/main/tests/test_transforms/test_smart_crusher_ccr_roundtrip.py):

```python
def roundtrip():
    # Compress a large JSON blob

    result = client.compress({"data": "..." * 1000})
    # The result includes a CCR key

    key = result.ccr_key

    # Retrieve the original via the tool

    recovered = client.headroom_retrieve(hash=key)

    assert recovered == {"data": "..." * 1000}

```

## Summary

- **CCR reversible compression** stores original content in a local cache keyed by a deterministic hash, replacing it with a lightweight placeholder to achieve 90%+ token savings.
- The **`headroom_retrieve`** tool (defined in [`headroom/tools.json`](https://github.com/chopratejas/headroom/blob/main/headroom/tools.json) and handled by [`headroom/ccr/response_handler.py`](https://github.com/chopratejas/headroom/blob/main/headroom/ccr/response_handler.py)) allows the LLM to recover stored content on demand.
- The **CCRResponseHandler** automatically detects retrieval requests, fetches data from the CCR store, and returns it as a tool response without client-side HTTP overhead.
- Optional **BM25-style queries** support searching within stored content before retrieval.
- The end-to-end workflow is validated by [`tests/test_transforms/test_smart_crusher_ccr_roundtrip.py`](https://github.com/chopratejas/headroom/blob/main/tests/test_transforms/test_smart_crusher_ccr_roundtrip.py), ensuring lossless compression and retrieval.

## Frequently Asked Questions

### What does CCR stand for in Headroom?

CCR stands for **Compress-Cache-Retrieve**. It describes the three-phase process where content is compressed by moving it to a cache, replaced with a marker in the prompt, and retrieved later when needed.

### How does the LLM know when to call headroom_retrieve?

The LLM receives a compression summary (generated by [`headroom/transforms/compression_summary.py`](https://github.com/chopratejas/headroom/blob/main/headroom/transforms/compression_summary.py)) that includes CCR markers with the `ccr_key`. When the model needs the original content to answer a question, it recognizes these markers and automatically emits a `headroom_retrieve` tool call with the appropriate hash.

### Is the CCR store persistent or in-memory?

According to the source implementation, the CCR store operates as a **local in-memory cache** during the session. The stored content persists long enough for the LLM to request it via `headroom_retrieve`, but the specific storage backend can be configured depending on the deployment environment.

### Can I search within stored content using headroom_retrieve?

Yes. The `headroom_retrieve` tool accepts an optional `query` parameter that performs a **BM25-style search** within the stored payload before returning content. This allows the LLM to retrieve only relevant sections (e.g., specific error messages from a large log file) rather than the entire original document.