# Main Modules and Components of Headroom: Context Compression Architecture Explained

> Explore Headroom's ten core modules like Transform, CacheAligner, and CCR storage. Understand its context compression architecture that sits between AI agents and LLM providers.

- Repository: [Tejas Chopra/headroom](https://github.com/chopratejas/headroom)
- Tags: architecture
- Published: 2026-06-21

---

**Headroom comprises ten specialized modules—including the Transform pipeline, CacheAligner, ContentRouter, CCR reversible storage, Proxy server, MCP server, Cross-Agent Memory, Telemetry, CLI, and Tokenizers—that together form a context-compression layer sitting between AI agents and LLM providers.**

Headroom is an open-source context-compression system developed in the `chopratejas/headroom` repository that reduces token costs for LLM interactions. Acting as a library, proxy, and MCP server, it processes prompts through a sophisticated transform pipeline before they reach providers like Anthropic or OpenAI. Understanding the main modules of headroom reveals how the system achieves reversible compression while maintaining cache efficiency and cross-agent memory.

## Core Transform Pipeline

The heart of headroom is the **transform pipeline**, implemented in [`headroom/transforms/pipeline.py`](https://github.com/chopratejas/headroom/blob/main/headroom/transforms/pipeline.py), which orchestrates a three-stage process: `CacheAligner` → `ContentRouter` → `Compressor`. This pipeline detects content types, applies specialized compression algorithms, and stabilizes message prefixes for optimal LLM cache utilization.

### CacheAligner

The `CacheAligner` class in [`headroom/transforms/cache_aligner.py`](https://github.com/chopratejas/headroom/blob/main/headroom/transforms/cache_aligner.py) stabilizes the prefix of message lists to ensure provider-side KV caches hit reliably. By trimming or padding the initial portion of conversations, it creates consistent cache keys across similar requests, reducing redundant computation on the LLM provider's side.

### ContentRouter

The `ContentRouter` in [`headroom/transforms/content_router.py`](https://github.com/chopratejas/headroom/blob/main/headroom/transforms/content_router.py) inspects each incoming message to determine its content type—whether JSON, code, plain text, or images. Based on this detection, it forwards the content to the appropriate compression engine. This routing logic ensures that structured data receives algorithmic compression while text may use model-based approaches.

### Compression Engines

Headroom includes specialized compressors accessed through the transform system:

- **SmartCrusher**: Located in [`headroom/transforms/smart_crusher.py`](https://github.com/chopratejas/headroom/blob/main/headroom/transforms/smart_crusher.py), this module handles JSON-centric compression using custom algorithms optimized for structured data.
- **CodeCompressor**: Found in [`headroom/transforms/code_compressor.py`](https://github.com/chopratejas/headroom/blob/main/headroom/transforms/code_compressor.py), this performs AST-aware compression for Python, JavaScript, Go, and other programming languages, preserving semantic structure while reducing token count.
- **Kompress-base**: A HuggingFace-based text compressor referenced in the architecture for general text compression tasks.

The public API for these transforms is lazily loaded through [`headroom/transforms/__init__.py`](https://github.com/chopratejas/headroom/blob/main/headroom/transforms/__init__.py).

## Storage and Retrieval Systems

### CCR (Reversible Compression)

The CCR system in [`headroom/ccr.py`](https://github.com/chopratejas/headroom/blob/main/headroom/ccr.py) stores original uncompressed payloads locally and provides reversible retrieval capabilities. When the pipeline compresses content, CCR maintains a mapping between compression markers and the full original data, enabling the `headroom_retrieve` endpoint to restore exact content on demand.

```python
from headroom.ccr import retrieve_original

# Retrieve the original payload using a compression token

original = retrieve_original("abc123")
print(original)   # Full, uncompressed content

```

### Cross-Agent Memory

The shared memory implementation in [`headroom/subscription/memory_handler.py`](https://github.com/chopratejas/headroom/blob/main/headroom/subscription/memory_handler.py) (also accessible via [`headroom/proxy/memory_handler.py`](https://github.com/chopratejas/headroom/blob/main/headroom/proxy/memory_handler.py)) deduplicates and indexes compressed contexts across multiple agents including Claude, Codex, and Gemini. This cross-agent store prevents repeated token usage when identical contexts appear in different agent conversations.

## Deployment Interfaces

Headroom exposes three primary deployment modes, each implemented as distinct modules:

### Python Library API

The public library interface is exposed through [`headroom/__init__.py`](https://github.com/chopratejas/headroom/blob/main/headroom/__init__.py), providing the `compress()` function that orchestrates the entire pipeline programmatically.

```python
from headroom import compress

messages = [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user",   "content": "Show me a Python function that flattens a dict."},
]

# Compress before sending to the LLM

compressed = compress(messages, model="gpt-4o")
print(compressed)  # Dict with compressed messages field

```

### HTTP Proxy Server

The FastAPI-based proxy in [`headroom/proxy/server.py`](https://github.com/chopratejas/headroom/blob/main/headroom/proxy/server.py) acts as a zero-code-change HTTP gateway. It intercepts requests to LLM providers, runs the compression pipeline transparently, and returns compressed responses.

```bash

# Start the proxy on port 8787

headroom proxy --port 8787

# Configure clients to use the proxy

export ANTHROPIC_BASE_URL=http://localhost:8787
claude "Explain how Bloom filters work"

```

The proxy entry point can also be invoked directly: `python -m headroom.proxy.server --port 8787`.

### MCP Server Integration

The Model Context Protocol implementation in `headroom/mcp_registry/*.py` (including [`install.py`](https://github.com/chopratejas/headroom/blob/main/install.py)) exposes `headroom_compress`, `headroom_retrieve`, and `headroom_stats` RPC methods. This allows any MCP-compatible client to access compression services without direct library integration.

```bash

# Install the MCP server

headroom mcp install

# Compress via RPC

curl -X POST http://localhost:8000/headroom_compress \
     -H "Content-Type: application/json" \
     -d '{"messages": [...]}'

```

## Observability and Infrastructure

### Telemetry and Metrics

The `headroom/telemetry/*.py` modules provide comprehensive observability including cost tracking, Prometheus metrics, and Langfuse tracing. These components record token counts, latency measurements, and cost estimates for every compression operation.

### Tokenizers and Utilities

Provider-agnostic token counting is implemented in `headroom/tokenizers/*.py`, supporting tiktoken, HuggingFace, and Mistral tokenizers. Shared infrastructure utilities including hashing, marker generation, safe JSON handling, and timestamp utilities reside in [`headroom/utils.py`](https://github.com/chopratejas/headroom/blob/main/headroom/utils.py).

## Command-Line Interface

The CLI implementation in `headroom/cli/*.py` provides user-facing commands that glue the library and proxy together. Key commands include `headroom wrap` for batch processing, `headroom proxy` for starting the gateway, and `headroom learn` for training or optimization tasks.

## Summary

- **Transform Pipeline**: The core [`headroom/transforms/pipeline.py`](https://github.com/chopratejas/headroom/blob/main/headroom/transforms/pipeline.py) orchestrates `CacheAligner` → `ContentRouter` → `Compressor` stages.
- **Compression Engines**: `SmartCrusher` (JSON) and `CodeCompressor` (AST) in the transforms directory handle type-specific compression.
- **CCR Storage**: [`headroom/ccr.py`](https://github.com/chopratejas/headroom/blob/main/headroom/ccr.py) provides reversible compression with original payload retrieval via `retrieve_original()`.
- **Cross-Agent Memory**: [`headroom/subscription/memory_handler.py`](https://github.com/chopratejas/headroom/blob/main/headroom/subscription/memory_handler.py) enables context sharing across different AI agents.
- **Deployment Modes**: Library API ([`headroom/__init__.py`](https://github.com/chopratejas/headroom/blob/main/headroom/__init__.py)), Proxy server ([`headroom/proxy/server.py`](https://github.com/chopratejas/headroom/blob/main/headroom/proxy/server.py)), and MCP server (`headroom/mcp_registry/*.py`).
- **Observability**: Telemetry modules track costs and performance metrics.
- **CLI Tools**: `headroom/cli/*.py` provides convenient command-line access to all functionality.

## Frequently Asked Questions

### What is the execution order of headroom's transform pipeline?

The pipeline executes in three sequential stages defined in [`headroom/transforms/pipeline.py`](https://github.com/chopratejas/headroom/blob/main/headroom/transforms/pipeline.py): first the `CacheAligner` stabilizes message prefixes for KV-cache optimization, then the `ContentRouter` detects content types, and finally the appropriate compressor (such as `SmartCrusher` or `CodeCompressor`) reduces the token payload.

### How does headroom maintain access to original uncompressed data?

The CCR (Reversible Compression) system in [`headroom/ccr.py`](https://github.com/chopratejas/headroom/blob/main/headroom/ccr.py) stores a local mapping between compression markers and original payloads. When compression occurs, the system generates a token (such as `abc123`) that can later be passed to `retrieve_original()` to fetch the complete, uncompressed content without loss.

### Which module handles cross-agent context sharing?

The cross-agent memory functionality is implemented in [`headroom/subscription/memory_handler.py`](https://github.com/chopratejas/headroom/blob/main/headroom/subscription/memory_handler.py). This module deduplicates compressed contexts across multiple agents—including Claude, Codex, and Gemini—preventing redundant token usage when identical contexts appear in different agent conversations.

### Can headroom be deployed without modifying existing code?

Yes, the [`headroom/proxy/server.py`](https://github.com/chopratejas/headroom/blob/main/headroom/proxy/server.py) module provides a FastAPI-based HTTP proxy that requires zero code changes. By setting the LLM provider's base URL to the proxy endpoint (e.g., `http://localhost:8787`), existing applications automatically benefit from compression while maintaining their original API calls to providers like Anthropic or OpenAI.