# How the headroom_retrieve Tool Works with CCR in Headroom

> Learn how headroom_retrieve works with CCR to complete the Compress-Cache-Retrieve loop, enabling LLMs to access full original data from a local cache while keeping proprietary information secure.

- Repository: [Tejas Chopra/headroom](https://github.com/chopratejas/headroom)
- Tags: how-to-guide
- Published: 2026-06-12

---

**The headroom_retrieve tool enables LLMs to request the full original data that was compressed away by accessing a local cache keyed by hash, completing the Compress-Cache-Retrieve (CCR) loop while keeping proprietary data local.**

The `headroom_retrieve` tool is a core component of the CCR (Compress-Cache-Retrieve) architecture in the [chopratejas/headroom](https://github.com/chopratejas/headroom) repository. It allows large language models to recover complete JSON payloads, code fragments, or log entries that were replaced with compression summaries during the initial request processing. This reversible compression system ensures that sensitive data never leaves the host machine while maintaining full context availability for the LLM.

## Understanding the CCR Architecture

Headroom’s CCR system makes every compression operation reversible. When the `compress()` function processes a message, it routes content through the **ContentRouter** → **SmartCrusher** / **CodeCompressor** / **Kompress-base** pipeline. The original payload is stored in a local cache keyed by a short hash, and only a compression summary is sent to the LLM.

The LLM receives the summary plus a tool definition for `headroom_retrieve`. When the model encounters a reference like "60 messages dropped, retrieve: def456", it can invoke the tool to fetch the original content from the local **compression_store.py**.

## How headroom_retrieve Completes the Loop

The CCR round-trip consists of four distinct phases handled by separate modules in the codebase.

### Step 1: Compression and Caching

In [`headroom/transforms/compression_summary.py`](https://github.com/chopratejas/headroom/blob/main/headroom/transforms/compression_summary.py), the compression engine processes the input and stores the original JSON in [`headroom/cache/compression_store.py`](https://github.com/chopratejas/headroom/blob/main/headroom/cache/compression_store.py). The function returns a hash key that uniquely identifies the cached content:

```python
from headroom import compress

# Compression returns both summary and metadata

compressed, meta = compress(messages, model="kompress-v2-base")
hash_key = meta["ccr_hash"]  # Stored in local CCR cache

```

The original data remains on the local machine at this stage.

### Step 2: Tool Injection

The [`headroom_ccr/tool_injection.py`](https://github.com/chopratejas/headroom/blob/main/headroom_ccr/tool_injection.py) module registers the `headroom_retrieve` tool and injects it into the request body’s `tools` array. This ensures the LLM knows it can request decompression:

```json
{
  "name": "headroom_retrieve",
  "arguments": {
    "hash": "<hash>",
    "query": "optional search term"
  }
}

```

The tool definition is exposed to the model before any generation begins.

### Step 3: LLM Invocation

When the model encounters a compression marker in the summary, it emits a `headroom_retrieve` tool call containing the hash. The proxy or MCP server intercepts this call before it reaches external APIs.

### Step 4: Response Handling and Retrieval

The [`headroom/ccr/response_handler.py`](https://github.com/chopratejas/headroom/blob/main/headroom/ccr/response_handler.py) module detects the tool call and fetches the stored JSON using `store.retrieve(hash_key)`. If a query parameter is provided, the handler runs a **BM25 search** over the original content and returns only matching snippets:

```python

# Full retrieval

store.retrieve("def456")

# BM25 search inside cached content

store.retrieve("def456", query="error traceback")

```

The retrieved content is returned as a normal message block to the LLM, completing the cycle.

## Implementation Details

The CCR system relies on specific source files to maintain the compression-retrieval loop:

- **[`headroom/ccr/tool_injection.py`](https://github.com/chopratejas/headroom/blob/main/headroom/ccr/tool_injection.py)**: Defines the tool schema and injects it into LLM requests.
- **[`headroom/ccr/response_handler.py`](https://github.com/chopratejas/headroom/blob/main/headroom/ccr/response_handler.py)**: Processes `headroom_retrieve` calls, interfaces with the cache, and handles BM25 queries.
- **[`headroom/cache/compression_store.py`](https://github.com/chopratejas/headroom/blob/main/headroom/cache/compression_store.py)**: Persistent key-value store mapping hashes to original JSON payloads.
- **[`headroom/ccr/mcp_server.py`](https://github.com/chopratejas/headroom/blob/main/headroom/ccr/mcp_server.py)**: Exposes the tool via the Model Context Protocol for non-proxy clients.
- **[`headroom/proxy/server.py`](https://github.com/chopratejas/headroom/blob/main/headroom/proxy/server.py)**: Intercepts tool calls in proxy mode, resolving them locally without client awareness.

The retrieval events are recorded for **TOIN learning** (Token Optimization via Inverse Narrative), allowing future compression decisions to improve based on which content the LLM actually requested. This feedback loop is tested in [`tests/test_toin_feedback.py`](https://github.com/chopratejas/headroom/blob/main/tests/test_toin_feedback.py).

## Practical Usage Examples

### Python Library Usage

Use the `retrieve` function directly after compression:

```python
from headroom import compress, retrieve

# Compress a large payload

compressed, meta = compress(large_messages, model="kompress-v2-base")
hash_key = meta["ccr_hash"]

# Later retrieval when LLM needs details

full_payload = retrieve(hash_key)
print(full_payload)

```

### MCP Server Integration

Install the MCP server to expose tools to any compatible client:

```bash
headroom mcp install

```

The server exposes three tools: `headroom_compress`, `headroom_retrieve`, and `headroom_stats`. When a client calls `headroom_retrieve(hash="abc123")`, the handler in [`headroom/ccr/mcp_server.py`](https://github.com/chopratejas/headroom/blob/main/headroom/ccr/mcp_server.py) resolves the request locally.

### Proxy Mode for OpenAI-Compatible Clients

Run the local proxy to automatically inject and handle retrieval:

```bash
headroom proxy --port 8787

```

The proxy defined in [`headroom/proxy/server.py`](https://github.com/chopratejas/headroom/blob/main/headroom/proxy/server.py) intercepts tool calls transparently, allowing any OpenAI-compatible client to benefit from CCR without code changes.

### Targeted Retrieval with BM25

For large log files or extensive JSON objects, retrieve only relevant sections:

```python

# Search for specific terms within the cached original

matches = retrieve(hash_key, query="authentication failed")

```

This runs the BM25 ranking algorithm in [`headroom/ccr/response_handler.py`](https://github.com/chopratejas/headroom/blob/main/headroom/ccr/response_handler.py) to return only the most relevant fragments.

## Summary

- **Local-First Security**: The `headroom_retrieve` tool ensures original data never leaves the host machine, making CCR safe for proprietary code and logs.
- **Hash-Based Retrieval**: Content is keyed by short hashes generated during compression and stored in [`headroom/cache/compression_store.py`](https://github.com/chopratejas/headroom/blob/main/headroom/cache/compression_store.py).
- **BM25 Search Support**: Optional query parameters enable the LLM to retrieve only specific fragments rather than entire payloads.
- **Multiple Interfaces**: The tool works via Python library, MCP server, or transparent proxy mode.
- **Learning Integration**: Retrieval patterns feed into TOIN feedback loops to improve future compression decisions.

## Frequently Asked Questions

### What happens if the hash is not found in the CCR cache?

If `headroom_retrieve` is called with an invalid or expired hash, [`headroom/ccr/response_handler.py`](https://github.com/chopratejas/headroom/blob/main/headroom/ccr/response_handler.py) returns an empty result or error message to the LLM. The cache operates locally, so hashes are only valid for the duration of the session or until explicitly cleared, depending on the configuration in [`headroom/cache/compression_store.py`](https://github.com/chopratejas/headroom/blob/main/headroom/cache/compression_store.py).

### Can the LLM search inside compressed content without retrieving everything?

Yes. By providing the optional `query` parameter in the tool call, the LLM triggers a BM25 search over the original cached content. The [`headroom/ccr/response_handler.py`](https://github.com/chopratejas/headroom/blob/main/headroom/ccr/response_handler.py) module ranks all fragments and returns only the top matches, allowing targeted retrieval of log entries or code snippets without loading the entire payload.

### Does using headroom_retrieve send data to external APIs?

No. The retrieval process is entirely local. The proxy or MCP server intercepts the tool call before it reaches any external API, looks up the hash in the local cache, and returns the content directly. This architecture ensures that proprietary data remains on the host machine even during the retrieval phase.

### How does the LLM know when to call headroom_retrieve?

The compression summary generated by [`headroom/transforms/compression_summary.py`](https://github.com/chopratejas/headroom/blob/main/headroom/transforms/compression_summary.py) includes explicit markers indicating which content was dropped and the corresponding hash (e.g., "60 messages dropped, retrieve: def456"). The LLM sees the `headroom_retrieve` tool in its available functions and can correlate these markers to invoke the tool when it needs the full original context.