Headroom Proxy Mode: Python SDK vs Framework Integrations Explained

The Headroom Python SDK connects to an external proxy process via HTTP, while framework integrations embed the proxy server directly into your ASGI application as an in-process middleware.

Headroom (chopratejas/headroom) is an open-source LLM proxy that optimizes API calls through prompt compression, CCR (Context Cost Reduction) injection, and telemetry tracking. According to the source code, you can access its proxy mode capabilities through two distinct architectural approaches: importing the lightweight Python SDK into standalone scripts, or mounting the proxy directly into web frameworks like FastAPI and Starlette.

The Python SDK Approach

Client Architecture

The Python SDK, implemented in headroom/client.py, provides the HeadroomClient class. This lightweight client serializes request payloads—including messages and tools—and forwards them to a running Headroom proxy via HTTP POST requests to /v1/chat/completions.

Key Capabilities

The client handles:

• Request signing and serialization
• Response parsing for JSON and SSE streaming
• Helper methods such as setup(), request(), ccr_len(), and ccr_get() for Context Cost Reduction operations
• Retry logic for failed connections

Configuration

The SDK respects the HEADROOM_SDK=proxy environment variable by default and requires HEADROOM_PROXY_URL to locate the proxy process:

from headroom.client import HeadroomClient

client = HeadroomClient(base_url="http://localhost:8000/v1")

Framework Integrations

Embedded ASGI Server

Framework integrations use the create_app() function from headroom/proxy/server.py to instantiate a HeadroomProxy as a standalone ASGI application. This mounts directly into FastAPI, Starlette, or other ASGI frameworks using standard mounting patterns like app.mount("/v1", proxy_app).

Request Handling Pipeline

When mounted, the integration:

• Intercepts incoming HTTP requests through provider-specific handlers in headroom/proxy/handlers/openai.py and headroom/proxy/handlers/anthropic.py
• Converts framework request objects into the proxy's internal model
• Executes the compression pipeline defined in headroom/proxy/compression_decision.py
• Returns responses in the framework's native format (e.g., FastAPI Response)

Configuration

Unlike the SDK, framework integrations use the ProxyConfig dataclass and accept flags such as --telemetry, --no-telemetry, --port, and --host:

from headroom.proxy.server import ProxyConfig, create_app

config = ProxyConfig(host="0.0.0.0", port=8000, telemetry=True)
proxy_app = create_app(config)

Side-by-Side Comparison

Aspect	Python SDK	Framework Integration
Process Model	External daemon required	In-process ASGI app
Source File	headroom/client.py	headroom/proxy/server.py
Entry Point	HeadroomClient	create_app() with ProxyConfig
Provider Logic	Client-side serialization	Server-side handlers in headroom/proxy/handlers/*.py
Telemetry	Forwards metadata to proxy	Auto-registers Prometheus metrics via headroom/proxy/prometheus_metrics.py
Best For	Scripts, notebooks, CLI tools	Production web services

Code Implementation Examples

Using the Python SDK

This approach requires running headroom proxy --port 8000 in a separate terminal. The HeadroomClient from headroom/client.py handles all communication:

from headroom.client import HeadroomClient

# Requires running: headroom proxy --port 8000

client = HeadroomClient(base_url="http://localhost:8000/v1")

messages = [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "Explain the difference between SDK and framework integrations."},
]

response = client.chat_completion(
    model="gpt-4o-mini",
    messages=messages,
    stream=False,
)

print(response.choices[0].message["content"])

FastAPI Integration

Mount the proxy as a sub-application using create_app() from headroom/proxy/server.py. This embeds the proxy directly into your FastAPI process:

from fastapi import FastAPI
from headroom.proxy.server import ProxyConfig, create_app

app = FastAPI()

config = ProxyConfig(
    host="0.0.0.0",
    port=8000,
    telemetry=True,
)

proxy_app = create_app(config)
app.mount("/headroom", proxy_app)

@app.get("/ping")
async def ping():
    return {"msg": "pong"}

Starlette Integration

For lightweight ASGI applications, mount the proxy using Starlette's routing system:

from starlette.applications import Starlette
from starlette.routing import Mount
from headroom.proxy.server import ProxyConfig, create_app

config = ProxyConfig(port=8001, telemetry=False)
proxy_app = create_app(config)

app = Starlette(routes=[
    Mount("/proxy", app=proxy_app),
])

Summary

• Headroom proxy mode provides LLM optimization through an intermediary layer that handles compression and cost tracking.
• The Python SDK (headroom/client.py) acts as a client-side wrapper requiring a separate proxy process, ideal for scripts and ad-hoc usage.
• Framework integrations embed the proxy server-side using create_app() from headroom/proxy/server.py, running in-process with your web application.
• Choose the SDK when you need flexibility in standalone scripts; choose framework integrations for production web services requiring automatic request routing.
• Both methods support the full feature set including CCR injection, compression decision logic, and telemetry collection.

Frequently Asked Questions

Can I use the Python SDK without running a separate proxy process?

No. The Python SDK requires a running Headroom proxy instance because HeadroomClient communicates via HTTP/HTTPS to the external process. For in-process execution without a separate daemon, use the framework integration approach with create_app() instead.

Does the FastAPI integration support all Headroom features?

Yes. When mounted via app.mount(), the integration utilizes the full proxy pipeline including provider-specific handlers in headroom/proxy/handlers/openai.py and headroom/proxy/handlers/anthropic.py, compression logic, and Prometheus metrics. The framework integration is functionally equivalent to running the standalone proxy.

How do I switch between OpenAI and Anthropic providers in the SDK?

The HeadroomClient automatically routes requests based on the model parameter and endpoint configuration. The proxy handlers in headroom/proxy/handlers/ translate the standardized requests to provider-specific formats, so you can use the same chat_completion method for both OpenAI and Anthropic APIs.

Which approach offers better performance for high-traffic applications?

Framework integrations generally offer lower latency for web applications because they eliminate the network hop between the SDK and proxy process. The in-process ASGI integration in headroom/proxy/server.py handles requests internally, while the Python SDK must serialize requests and send them over HTTP to the external proxy daemon.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}