How to Use Headroom in Proxy Mode for Zero-Code Integration

Headroom's stand-alone HTTP proxy lets you compress LLM context by simply changing the base URL in your client—no code changes required.

The chopratejas/headroom repository provides a production-ready proxy that intercepts requests between your LLM client and upstream providers (Anthropic, OpenAI, Vertex AI). By deploying Headroom in proxy mode, you enable intelligent context compression through environment variable configuration alone, eliminating the need to import SDKs or refactor existing codebases.

How the Proxy Architecture Works

Headroom ships as a Rust-compiled binary wrapped in a FastAPI/Uvicorn application (located in headroom/proxy/server.py). The proxy sits transparently between any HTTP client and the LLM provider, executing the full compression pipeline before forwarding optimized requests upstream.

The IntelligentContextManager Pipeline

When a request hits the proxy, it triggers the IntelligentContextManager pipeline as defined in wiki/ARCHITECTURE.md. This pipeline sequences through:

• SmartCrusher: Identifies redundant system messages and duplicate content
• ContentRouter: Scores messages by recency, semantic similarity, and TOIN (Token-Optimized Information Notation) patterns
• CacheAligner: Manages prefix caching strategy for optimal KV-cache reuse

The proxy returns a compressed request to the provider, with all heavy-weight logic handled server-side. According to the source in wiki/proxy.md, the proxy can also expose a pure-compression endpoint at POST /v1/compress (used by the TypeScript SDK), enabling "compression-as-a-service" for any HTTP client.

Key Architectural Components

Component	Source Location	Function
Proxy Server	headroom/proxy/server.py	FastAPI app handling routing, health endpoints, metrics, and the /v1/compress endpoint
Semantic Cache	wiki/proxy.md – Semantic Caching section	LRU-evicted cache with TTL, keyed by hash of message lists
CCR Integration	wiki/proxy.md – CCR Integration	Persists dropped messages for later retrieval by the LLM
Budget Enforcement	wiki/proxy.md – Cost Tracking	Real-time USD cost estimation and daily budget caps
Telemetry	wiki/proxy.md – OTEL section	Optional OpenTelemetry export and Prometheus metrics

Installation and Setup

Installing the Proxy Extra

The proxy requires additional dependencies including the compiled Rust binary, SQLite/HNSW memory store, and optional OpenTelemetry components:

# Core proxy only

pip install "headroom-ai[proxy]"

# With OpenTelemetry observability

pip install "headroom-ai[proxy,otel]"

Running the Proxy Server

Start the proxy with default settings (binds to 127.0.0.1:8787):

headroom proxy

For production configurations, specify host, port, logging, and budget constraints:

headroom proxy \
    --host 0.0.0.0 \
    --port 8787 \
    --log-file /var/log/headroom.jsonl \
    --budget 100.0

Full CLI options are documented in wiki/proxy.md under the Core Options section.

Zero-Code Integration Steps

Integration requires only environment variable changes—no source code modifications.

Configure Claude Code (Anthropic)

Set the ANTHROPIC_BASE_URL environment variable to point at your running proxy:

export ANTHROPIC_BASE_URL=http://localhost:8787
claude  # Runs with automatic context compression

Configure GitHub Copilot CLI

Use the headroom wrap command for Copilot integration:

headroom wrap copilot -- --model claude-sonnet-4-20250514

Configure OpenAI-Compatible SDKs

Point any OpenAI-compatible client (Python, Node.js, etc.) to the proxy endpoint:

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:8787/v1",  # Proxy endpoint

    api_key="your-api-key"                # Still required for upstream auth

)

Direct Compression Endpoint

Custom HTTP clients can call the compression endpoint directly without invoking an LLM:

curl -X POST http://localhost:8787/v1/compress \
     -H "Content-Type: application/json" \
     -d '{"messages":[...],"model":"gpt-4o"}'

Run Modes and Performance Tuning

Headroom proxy supports two run modes that trade off between immediate token reduction and cache reuse stability, as documented in wiki/proxy.md:

Token mode (--mode token): Maximizes immediate token savings by aggressively compressing context. Ideal for stateless interactions.

headroom proxy --mode token

Cache mode (HEADROOM_MODE=cache): Preserves prior conversation turns to optimize prefix-cache reuse. Critical for long-running agents maintaining state across multiple requests.

HEADROOM_MODE=cache headroom proxy

Observability and Telemetry

Enable OpenTelemetry and Prometheus metrics by setting environment variables before launching the proxy:

export HEADROOM_OTEL_METRICS_ENABLED=1
export HEADROOM_OTEL_METRICS_EXPORTER=otlp_http
export HEADROOM_OTEL_METRICS_ENDPOINT=http://127.0.0.1:4318/v1/metrics
export HEADROOM_OTEL_SERVICE_NAME=headroom-proxy

For local debugging, use the console exporter:

export HEADROOM_OTEL_METRICS_EXPORTER=console

Metric names follow the conventions documented in wiki/metrics.md, including headroom_requests_total and headroom_tokens_saved_total.

Production Deployment

Deploy under a process manager or using Docker. The repository provides a production-ready Dockerfile:

FROM python:3.11-slim
RUN apt-get update && apt-get install -y --no-install-recommends build-essential \
    && pip install "headroom-ai[proxy]" \
    && apt-get purge -y build-essential && apt-get autoremove -y \
    && rm -rf /var/lib/apt/lists/*
EXPOSE 8787
CMD ["headroom", "proxy", "--host", "0.0.0.0"]

The complete Dockerfile is available in the repository root as referenced in wiki/proxy.md.

Summary

• Headroom in proxy mode provides zero-code integration by acting as a transparent HTTP proxy between your LLM client and provider
• The IntelligentContextManager pipeline (headroom/proxy/server.py) handles compression via SmartCrusher, ContentRouter, and CacheAligner components
• Install using pip install "headroom-ai[proxy]" and run with headroom proxy
• Configure clients via environment variables (ANTHROPIC_BASE_URL, OPENAI_BASE_URL) or the headroom wrap command
• Choose between token mode (maximum immediate savings) and cache mode (prefix-cache optimization for long-running agents)
• Enable OpenTelemetry and Prometheus metrics for production observability

Frequently Asked Questions

What is the difference between token mode and cache mode in Headroom proxy?

Token mode (--mode token) maximizes immediate token reduction by aggressively compressing the context window, while cache mode (HEADROOM_MODE=cache) preserves conversation structure to optimize prefix-cache reuse across turns. Use token mode for stateless requests and cache mode for long-running agents that benefit from KV-cache persistence.

Do I need to modify my existing LLM client code to use Headroom proxy?

No. Headroom proxy enables zero-code integration by implementing the standard OpenAI/Anthropic HTTP API. Simply change your client's base URL environment variable (e.g., ANTHROPIC_BASE_URL or OPENAI_BASE_URL) to point to http://localhost:8787, and the proxy will automatically compress requests before forwarding them to the upstream provider.

How does the semantic cache work in proxy mode?

The semantic cache stores compressed responses keyed by a hash of the message list, as implemented in wiki/proxy.md. It uses LRU eviction and configurable TTL to serve identical or highly similar requests without hitting the upstream provider. Dropped messages are persisted via the CCR (Content-Cache-Relay) integration so the LLM can retrieve them if needed during the conversation.

Can I use Headroom proxy with custom HTTP clients that aren't OpenAI-compatible?

Yes. Any HTTP client can use the compression service directly by calling POST /v1/compress at the proxy endpoint. This returns a compressed context payload without invoking an LLM, allowing custom integrations to benefit from Headroom's IntelligentContextManager pipeline regardless of the final destination API format.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}