How Headroom Supports Different LLM Providers: OpenAI, Anthropic, Google Vertex AI, and Bedrock

Headroom unifies OpenAI, Anthropic, Google Vertex AI, and AWS Bedrock under a single abstract Provider interface, enabling seamless token counting, context limit enforcement, and cost estimation across all backends through a pluggable registry system.

Headroom is an open-source LLM proxy that abstracts vendor-specific implementations into a consistent interface. By treating each LLM provider as a modular plugin that implements the Provider base class in headroom/providers/base.py, Headroom eliminates backend-specific code from your application while maintaining accurate telemetry across diverse model endpoints.

The Provider Interface

Every LLM backend in Headroom must implement the abstract Provider class defined in headroom/providers/base.py. This contract ensures uniform behavior regardless of the underlying vendor.

The base class defines five required methods:

class Provider(ABC):
    @property
    @abstractmethod
    def name(self) -> str: ...

    @abstractmethod
    def supports_model(self, model: str) -> bool: ...

    @abstractmethod
    def get_token_counter(self, model: str) -> TokenCounter: ...

    @abstractmethod
    def get_context_limit(self, model: str) -> int: ...

    @abstractmethod
    def estimate_cost(
        self,
        input_tokens: int,
        output_tokens: int,
        model: str,
        cached_tokens: int = 0,
    ) -> float | None: ...

Concrete provider classes in headroom/providers/openai.py, headroom/providers/anthropic.py, headroom/providers/google.py, and headroom/providers/litellm.py inherit this interface, guaranteeing that the proxy layer can query token counts, validate context windows, and calculate costs without knowing which vendor serves the request.

OpenAI Provider Implementation

The OpenAIProvider class in headroom/providers/openai.py handles GPT-4, GPT-4o, and other OpenAI-compatible endpoints.

Token counting uses tiktoken with model-specific encodings defined in the _MODEL_ENCODINGS dictionary. Context limits resolve through a priority chain: LiteLLM metadata first, then explicit constructor limits, followed by the HEADROOM_MODEL_LIMITS environment variable, ~/.headroom/models.json configuration, built-in _CONTEXT_LIMITS, pattern inference, and finally a fallback default.

Cost estimation prefers LiteLLM pricing data when available, otherwise falls back to hard-coded _PRICING tables. The provider approximates cached-token discounts at 50% in the _estimate_cost_manual method.

from headroom.providers import ProviderRegistry

registry = ProviderRegistry()
openai = registry.get("openai")

model = "gpt-4o-mini"
messages = [
    {"role": "user", "content": "Explain quantum tunneling in one sentence."}
]

counter = openai.get_token_counter(model)
token_count = counter.count_messages(messages)
context_limit = openai.get_context_limit(model)
cost = openai.estimate_cost(
    input_tokens=token_count,
    output_tokens=0,
    model=model,
)

Anthropic Provider Implementation

The AnthropicProvider in headroom/providers/anthropic.py supports Claude 3 and later models with a hybrid token-counting strategy.

By default, the AnthropicTokenCounter uses tiktoken approximations, but when an anthropic.Anthropic client is supplied, it can call the official Anthropic Token Count API for exact measurements. Context limits are stored in ANTHROPIC_CONTEXT_LIMITS, with unknown models handled via tier inference logic in _infer_model_tier.

Cost estimation attempts LiteLLM first, then falls back to ANTHROPIC_PRICING tables for standard input/output rates.

from anthropic import Anthropic
from headroom.providers import ProviderRegistry

anthropic_client = Anthropic()
registry = ProviderRegistry()
anthropic = registry.get("anthropic")

model = "claude-3-sonnet-20241022"
counter = anthropic.get_token_counter(model)  # Uses API when client provided

token_count = counter.count_messages(messages)

Google Vertex AI Provider Implementation

The GoogleProvider in headroom/providers/google.py manages Gemini models hosted on Google Vertex AI.

Token counting defaults to cl100k_base encoding (the same as Claude models) via tiktoken. Context limits reference the static _GOOGLE_CONTEXT_LIMITS map, which mirrors current Gemini context windows. Cost estimation pulls from _GOOGLE_PRICING, with unknown models defaulting to Gemini-1-Pro-001 pricing tiers.

AWS Bedrock via LiteLLM

Headroom does not implement a native Bedrock provider. Instead, headroom/providers/litellm.py provides a LiteLLMProvider that wraps the LiteLLM library to support AWS Bedrock and other LiteLLM-compatible backends.

For Bedrock specifically:

• Model discovery and context limits query LiteLLM's get_model_info first; if unavailable, Headroom falls back to internal _BEDROCK_CONTEXT_LIMITS populated from Bedrock inference profiles
• Token counting delegates to the appropriate vendor-specific provider (OpenAI or Anthropic) once the Bedrock endpoint resolves
• Cost estimation uses LiteLLM's completion_cost when available, otherwise falls back to _BEDROCK_PRICING

Helper methods like _fetch_bedrock_inference_profiles and _normalize_bedrock_profile_id handle AWS-specific profile normalization.

# Configure via environment variables

import os
os.environ["HEADROOM_BACKEND"] = "bedrock"
os.environ["HEADROOM_BEDROCK_REGION"] = "us-east-1"

from headroom.providers import ProviderRegistry

registry = ProviderRegistry()
bedrock = registry.get("bedrock")  # LiteLLMProvider configured for Bedrock

limit = bedrock.get_context_limit("anthropic.claude-3-opus-20240229")

Provider Registration and Runtime Selection

The ProviderRegistry class in headroom/providers/registry.py instantiates and manages all providers:

class ProviderRegistry:
    def __init__(self):
        self._providers: dict[str, Provider] = {
            "openai": OpenAIProvider(),
            "anthropic": AnthropicProvider(),
            "google": GoogleProvider(),
            "bedrock": LiteLLMProvider(backend="bedrock"),
        }

    def get(self, kind: str) -> Provider:
        return self._providers[kind]

Selection happens at runtime through:

1. The HEADROOM_BACKEND environment variable
2. Request headers such as X-Headroom-Provider
3. Fallback to the OpenAI-compatible provider if the requested backend is unavailable

The HTTP proxy layer in headroom/proxy/handlers/ extracts these hints and retrieves the appropriate provider instance from the registry.

Configuration and Customization

You can override built-in limits and pricing without modifying source code using environment variables or configuration files.

Override model limits and pricing via HEADROOM_MODEL_LIMITS:

export HEADROOM_MODEL_LIMITS='{
  "openai": {
    "context_limits": {"my-gpt-4o-custom": 256000},
    "pricing": {"my-gpt-4o-custom": [3.0, 12.0]}
  }
}'

registry = ProviderRegistry()
openai = registry.get("openai")
print(openai.get_context_limit("my-gpt-4o-custom"))  # Returns 256000

Headroom also checks ~/.headroom/models.json for persistent custom model definitions, loaded during provider initialization in registry.py.

Summary

• Unified Interface: All providers implement the abstract Provider class in headroom/providers/base.py, exposing get_token_counter(), get_context_limit(), and estimate_cost() methods.
• Pluggable Backends: Concrete implementations in openai.py, anthropic.py, google.py, and litellm.py handle vendor-specific logic while presenting a common API.
• Bedrock Support: AWS Bedrock is supported through the LiteLLMProvider in litellm.py, which delegates token counting to underlying vendor providers and manages Bedrock-specific inference profiles.
• Runtime Selection: The ProviderRegistry in registry.py instantiates providers and routes requests based on HEADROOM_BACKEND environment variables or request headers.
• Extensible Configuration: Custom model limits and pricing can be injected via HEADROOM_MODEL_LIMITS or ~/.headroom/models.json without code changes.

Frequently Asked Questions

How does Headroom handle token counting for different LLM providers?

Each provider implements get_token_counter(), returning a vendor-specific TokenCounter instance. OpenAI uses tiktoken with model-specific encodings, Anthropic can use either tiktoken approximations or the official Anthropic Token Count API when a client is provided, Google uses cl100k_base encoding, and Bedrock delegates to the underlying provider's counter once the model is resolved.

Can I use custom model limits or pricing not built into Headroom?

Yes. Set the HEADROOM_MODEL_LIMITS environment variable with a JSON object containing context_limits and pricing dictionaries, or create a ~/.headroom/models.json file. The OpenAIProvider checks these sources in its resolution chain before falling back to built-in defaults.

How does Headroom route requests to the correct provider backend?

The ProviderRegistry in headroom/providers/registry.py maintains a dictionary of provider instances. At runtime, the proxy layer checks the HEADROOM_BACKEND environment variable or the X-Headroom-Provider request header to determine which provider to use, then calls registry.get() to retrieve the appropriate instance.

Does Headroom support AWS Bedrock natively or through an intermediary?

Headroom supports Bedrock through the LiteLLMProvider in headroom/providers/litellm.py, which wraps the LiteLLM library. This provider handles Bedrock-specific inference profile normalization via _fetch_bedrock_inference_profiles() and _normalize_bedrock_profile_id(), while delegating token counting to the underlying Anthropic or OpenAI providers depending on the Bedrock model being accessed.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}