How Is the Headroom Code Organized? Understanding the Python SDK Architecture

Headroom organizes its Python SDK into a modular package structure with lazy-loaded public APIs, a central HeadroomClient orchestration layer, provider-specific transforms, and separate subsystems for caching, tokenization, observability, and CLI tooling.

Headroom is a Python SDK that wraps LLM clients to provide context-budget control, smart compression, and observability. The headroom code organized in the chopratejas/headroom repository follows standard Python package conventions with clear separation of concerns across multiple submodules. This architecture enables independent development of tokenization, transformation, and caching logic while presenting a unified interface to developers.

Public API and Lazy Loading Strategy

The entry point headroom/__init__.py defines the public API surface using a lazy export mechanism. It registers a _LAZY_EXPORTS mapping that loads submodules only when attributes are first accessed, keeping import costs minimal.

When you import headroom, the package defers loading heavy dependencies until specific classes like HeadroomClient or OpenAIProvider are referenced.

The HeadroomClient Orchestration Layer

At the core of the SDK sits HeadroomClient in headroom/client.py. This class wraps any LLM client (OpenAI-style or Anthropic-style) and coordinates the entire request lifecycle.

Construction involves several steps:

• Receiving an underlying LLM client and a Provider implementation
• Building a HeadroomConfig instance
• Creating a storage backend for metrics
• Instantiating a TransformPipeline

The _create method handles request processing by parsing messages, computing cache-alignment scores, running the transform pipeline, checking semantic cache layers, and persisting RequestMetrics records.

Modular Transforms Pipeline

The TransformPipeline class in headroom/transforms/pipeline.py implements a modular processing system. Each transform lives under headroom/transforms/ and adheres to a common interface.

Key transforms include:

• SmartCrusher: Removes irrelevant content from requests
• CacheAligner: Trims prefix tokens to improve cache hit rates

Transforms execute in the order defined by CANONICAL_PIPELINE_STAGES, allowing composable request optimization.

Provider Abstraction and Tokenization

The SDK abstracts provider differences through headroom/providers/base.py, which defines interfaces for token counting, context limits, and transport calls. Concrete implementations handle provider-specific quirks while presenting a unified API.

Token counting utilities reside in headroom/tokenizer.py, wrapping provider-specific counters with helpers to count tokens in messages or raw text.

Caching and Optimization Subsystem

Cache optimization lives in headroom/cache/base.py with provider-specific optimizers like OpenAICacheOptimizer and AnthropicCacheOptimizer. The CacheOptimizerRegistry manages these implementations.

When enabled, the client builds an OptimizationContext and requests cache-control block insertion or response reuse. An optional semantic-cache overlay provides additional deduplication capabilities.

Observability, Storage, and CLI

The headroom/observability.py module provides OpenTelemetry and LangFuse integrations, emitting events like INPUT_RECEIVED, INPUT_COMPRESSED, and POST_SEND through the PipelineExtensionManager.

Request metrics persist via headroom/storage.py using SQLite or JSONL backends, storing RequestMetrics records with querying capabilities.

Command-line tools reside in headroom/cli/, with headroom/cli/main.py exposing SDK functionality through commands like headroom wrap, headroom install, and headroom perf.

Evaluation and Testing Infrastructure

The headroom/evals/ directory contains reusable evaluation suites for benchmarking transforms and cache optimizers, orchestrated by headroom/evals/suite_runner.py.

Unit tests in tests/ and synthetic benchmarks in benchmarks/ validate correctness and performance.

Practical Example

# Quick start – wrap an OpenAI client

from headroom import HeadroomClient, OpenAIProvider
from openai import OpenAI

client = HeadroomClient(
    original_client=OpenAI(),
    provider=OpenAIProvider(),
    default_mode="optimize",
)

# Standard OpenAI-style call (the SDK rewrites the payload internally)

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Explain headroom architecture"}],
)

print(response.choices[0].message.content)

# Simulate the optimisation without sending a request

sim = client.chat.completions.simulate(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Long text …"}],
)

print(f"Tokens saved: {sim.tokens_saved}")
print(f"Transforms applied: {sim.transforms}")

# Retrieve in-memory stats for the current session

stats = client.get_stats()
print(f"Session token savings: {stats['session']['tokens_saved_total']}")

Summary

• headroom/init.py uses lazy loading via _LAZY_EXPORTS to minimize import overhead
• headroom/client.py contains HeadroomClient, the central orchestration class wrapping LLM clients
• headroom/transforms/ houses the TransformPipeline and individual transforms like SmartCrusher and CacheAligner
• headroom/cache/ and headroom/providers/ provide provider-specific optimizations and abstractions
• headroom/storage.py and headroom/observability.py handle metrics persistence and tracing
• headroom/cli/ exposes SDK functionality as command-line tools

Frequently Asked Questions

How does Headroom minimize import time?

Headroom implements lazy loading in headroom/__init__.py through a _LAZY_EXPORTS mapping. This mechanism defers submodule loading until specific attributes are first accessed, ensuring that importing the package does not immediately load heavy dependencies or complex subsystems.

What is the role of the TransformPipeline?

The TransformPipeline class in headroom/transforms/pipeline.py executes a series of request modifications in the order defined by CANONICAL_PIPELINE_STAGES. It chains transforms like SmartCrusher for content removal and CacheAligner for token trimming, allowing modular composition of optimization strategies.

How does Headroom support different LLM providers?

The SDK abstracts provider specifics through the Provider base class in headroom/providers/base.py and the CacheOptimizerRegistry. Concrete implementations like OpenAICacheOptimizer handle provider-specific cache control insertion, while HeadroomClient manages transport calls and token counting uniformly across OpenAI-style and Anthropic-style APIs.

Where does Headroom store request metrics?

Request metrics persist through the storage backend defined in headroom/storage.py, which supports both SQLite and JSONL formats. The HeadroomClient creates RequestMetrics records during request processing and writes them to the configured store, enabling session-level statistics retrieval via client.get_stats().

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}