How Image Compression Works in Headroom: A Technical Deep Dive

Headroom reduces LLM token costs by automatically detecting images in API requests, routing them through a trained Mini‑LM classifier, and applying provider‑specific compression techniques—such as OCR transcoding or resolution downsizing—before forwarding the request.

The chopratejas/headroom repository implements an intelligent image compression pipeline that intercepts multimodal requests to minimize token usage. At its core, the ImageCompressor class orchestrates a zero‑loss tile optimization, a trained neural router, and provider‑specific transformations to reduce costs by 40–90% without sacrificing response quality.

Detecting Images in the Message Payload

The pipeline begins in headroom/image/compressor.py with the has_images method, which scans incoming message arrays for embedded images across OpenAI, Anthropic, and Google formats.

def has_images(self, messages: list[dict[str, Any]]) -> bool:
    for message in messages:
        content = message.get("content")
        if isinstance(content, list):
            for item in content:
                if isinstance(item, dict):
                    # OpenAI format

                    if item.get("type") == "image_url": return True
                    # Anthropic format

                    if item.get("type") == "image": return True
                    # Google format

                    if "inlineData" in item: return True

(see lines 74‑90 in [headroom/image/compressor.py](https://github.com/chopratejas/headroom/blob/main/headroom/image/compressor.py#L74-L90))

This detection handles the three provider‑specific image formats before any expensive processing occurs.

Extracting Query Context and Image Bytes

Before routing decisions are made, Headroom extracts the textual prompt accompanying the image and retrieves the raw byte data for analysis.

Extracting the User Query

The _extract_query method walks messages in reverse chronological order to find the most recent user text, which the router uses to determine intent.

def _extract_query(self, messages):
    for message in reversed(messages):
        if message.get("role") != "user": continue
        content = message.get("content")
        if isinstance(content, str): return content
        if isinstance(content, list):
            texts = [item.get("text", "") for item in content
                     if isinstance(item, dict) and item.get("type") == "text"]
            if texts: return " ".join(texts)
    return ""

(see lines 92‑117 in [headroom/image/compressor.py](https://github.com/chopratejas/headroom/blob/main/headroom/image/compressor.py#L92-L117))

Pulling Raw Image Bytes

The _extract_image_data method normalizes the various provider formats (data URLs, base64 sources, inlineData) into raw bytes for downstream OCR and embedding analysis.

def _extract_image_data(self, messages):
    for message in messages:
        content = message.get("content")
        if not isinstance(content, list): continue
        for item in content:
            if not isinstance(item, dict): continue
            # OpenAI – data URL

            if item.get("type") == "image_url" and (url := item.get("image_url", {}).get("url", "")).startswith("data:"):
                return base64.b64decode(re.match(r"data:image/[^;]+;base64,(.+)", url).group(1))
            # Anthropic – base64 source

            if item.get("type") == "image" and (src := item.get("source", {})).get("type") == "base64":
                return base64.b64decode(src.get("data", ""))
            # Google – inlineData

            if "inlineData" in item:
                return base64.b64decode(item["inlineData"].get("data", ""))
    return None

(see lines 119‑149 in [headroom/image/compressor.py](https://github.com/chopratejas/headroom/blob/main/headroom/image/compressor.py#L119-L149))

Pre‑Optimization: Tile‑Boundary Alignment

Before invoking the ML router, Headroom attempts a zero‑loss optimization by aligning image dimensions to the provider’s token grid. This can shave tokens without altering pixel data.

from .tile_optimizer import optimize_images_in_messages
messages, tile_results = optimize_images_in_messages(messages, provider)

(see lines 334‑339 in [headroom/image/compressor.py](https://github.com/chopratejas/headroom/blob/main/headroom/image/compressor.py#L334-L339))

If the image dimensions already line up with the provider’s token‑grid, the optimizer reduces token count computationally rather than visually.

Routing with Mini‑LM and SigLIP

The compression strategy is determined by a trained Mini‑LM classifier (fine‑tuned on 1,157 examples) and an optional SigLIP model for image‑property analysis.

Loading the Trained Router

The router is loaded lazily via _get_router to avoid unnecessary initialization overhead when processing text‑only requests.

def _get_router(self) -> TrainedRouter:
    if self._router is None:
        from .trained_router import TrainedRouter
        self._router = TrainedRouter(
            model_path=self.model_id,
            use_siglip=self.use_siglip,
            device=self.device,
        )
    return self._router

(see lines 154‑165 in [headroom/image/compressor.py](https://github.com/chopratejas/headroom/blob/main/headroom/image/compressor.py#L154-L165))

The TrainedRouter implementation lives in headroom/image/trained_router.py and encapsulates the classifier and SigLIP embedding logic.

Classifying Query Intent and Image Signals

The router’s classify method first predicts a technique based on the textual query (_classify_query). If use_siglip=True, it extracts image embeddings and computes signal scores (_analyze_image) to adjust the final decision.

technique, query_confidence = self._classify_query(query)
if self.use_siglip:
    image_embedding = self._get_image_embedding(image_data)
    image_signals = self._analyze_image(image_embedding)

# Apply heuristics → final RouteDecision

(see the classify method in [headroom/image/trained_router.py](https://github.com/chopratejas/headroom/blob/main/headroom/image/trained_router.py#L334-L388))

The router returns a RouteDecision containing:

• technique — one of TRANSCODE, CROP, FULL_LOW, or PRESERVE
• confidence — a probability‑like score indicating prediction certainty

Applying Compression Techniques

The _apply_compression method in headroom/image/compressor.py (lines 388‑460) transforms each image block based on the router’s decision.

TRANSCODE: OCR Replacement

When the router selects TRANSCODE, Headroom runs OCR via _ocr_extract and substitutes the image block with extracted text.

if technique.value == "transcode":
    extracted = self._ocr_extract(image_bytes_for_ocr)
    if extracted:
        new_content.append({"type": "text",
                            "text": f"[OCR from image]\n{extracted}"})

This eliminates image tokens entirely, replacing them with significantly cheaper text tokens.

FULL_LOW and CROP: Resolution Reduction

For FULL_LOW, CROP, or when transcoding fails, Headroom applies provider‑specific resizing:

OpenAI — Sets detail="low" in the image URL:

if provider == "openai":
    new_content.append({"type": "image_url",
                        "image_url": {..., "detail": "low"}})

Anthropic — Resizes to 512 px maximum dimension using Pillow’s Image.resize:

elif provider == "anthropic":
    resized, media_type = self._resize_image(image_bytes_for_ocr, max_dimension=512)
    new_content.append({"type": "image",
                        "source": {"type": "base64",
                                   "media_type": media_type,
                                   "data": base64.b64encode(resized).decode()}})

Google — Resizes to 768 px maximum dimension:

elif provider == "google":
    resized, media_type = self._resize_image(image_bytes_for_ocr, max_dimension=768)
    new_content.append({"inlineData": {"mimeType": media_type,
                                       "data": base64.b64encode(resized).decode()}})

(see _resize_image at lines 251‑285 in [headroom/image/compressor.py](https://github.com/chopratejas/headroom/blob/main/headroom/image/compressor.py#L251-L285))

PRESERVE: Pass‑Through

When confidence is low or the image requires full fidelity, the PRESERVE technique leaves the image untouched, forwarding the original bytes unchanged.

Token Accounting and Savings Calculation

Headroom estimates token costs before and after compression using provider‑specific formulas. The results are stored in a CompressionResult object attached to the compressor instance.

self.last_result = CompressionResult(
    technique=technique,
    original_tokens=original_tokens,
    compressed_tokens=compressed_tokens,
    confidence=confidence,
)

(see lines 708‑714 in [headroom/image/compressor.py](https://github.com/chopratejas/headroom/blob/main/headroom/image/compressor.py#L708-L714))

The last_savings property calculates the percentage reduction, typically ranging from 40 % to 90 % depending on the technique applied.

Usage Examples

Basic Image Compression

Instantiate the ImageCompressor class directly to process message arrays before forwarding to an LLM provider:

from headroom.image import ImageCompressor

compressor = ImageCompressor()
compressed = compressor.compress(messages, provider="openai")
print(f"Savings: {compressor.last_savings:.1f}%")
compressor.close()

One‑Shot Helper Function

For simpler use cases, use the compress_images helper which handles initialization and cleanup:

from headroom.image import compress_images

compressed = compress_images(messages, provider="anthropic")

(see implementation at lines 734‑751 in [headroom/image/compressor.py](https://github.com/chopratejas/headroom/blob/main/headroom/image/compressor.py#L734-L751))

Disabling Compression in Proxy Mode

When running the Headroom proxy server, disable image optimization entirely via CLI flag:

headroom proxy --no-image-optimize

This sets proxy.config.image_optimize = False, bypassing the ImageCompressor pipeline while preserving other proxy features.

Summary

• Headroom intercepts multimodal LLM requests in chopratejas/headroom to reduce token costs through intelligent image compression.
• The ImageCompressor class detects images in OpenAI, Anthropic, and Google formats using has_images (lines 74‑90).
• A Mini‑LM router (trained_router.py) analyzes query text and optional SigLIP embeddings to select one of four techniques: TRANSCODE, CROP, FULL_LOW, or PRESERVE.
• Zero‑loss tile optimization (lines 334‑339) attempts cheap token savings before ML routing.
• Provider‑specific resizing uses Pillow’s Image.resize (lines 251‑285) to downscale images to 512 px (Anthropic) or 768 px (Google), while OpenAI requests use the detail="low" parameter.
• OCR transcoding replaces image bytes with extracted text blocks, eliminating image tokens entirely.
• Token savings are calculated and stored in CompressionResult (lines 708‑714), with typical reductions of 40–90 %.

Frequently Asked Questions

What compression techniques does Headroom support?

Headroom supports four techniques defined in the Technique enum: TRANSCODE (OCR text replacement), CROP (smart region selection), FULL_LOW (resolution reduction), and PRESERVE (original image). The TRANSCODE technique eliminates image tokens entirely by substituting OCR‑extracted text, while FULL_LOW uses provider‑specific resizing (512 px for Anthropic, 768 px for Google, or detail="low" for OpenAI).

How does Headroom decide which compression technique to use?

A trained Mini‑LM classifier (TrainedRouter in headroom/image/trained_router.py) analyzes the user’s textual query to predict the appropriate technique. If the use_siglip option is enabled, the router also extracts image embeddings using SigLIP to compute signal scores (e.g., text density, visual complexity) that can override or adjust the initial prediction. The final RouteDecision includes both the selected technique and a confidence score.

Can I use Headroom image compression without the proxy server?

Yes. The ImageCompressor class and the compress_images helper function are designed for standalone use. Import them from headroom.image and call compressor.compress(messages, provider="...") before forwarding the modified payload to your LLM provider. This integration pattern works with direct API calls, serverless functions, or custom middleware without requiring the Headroom proxy.

What is the tile‑boundary optimization and when does it apply?

The tile‑boundary optimization is a zero‑loss preprocessing step (implemented in headroom/image/tile_optimizer.py) that aligns image dimensions to the provider’s token grid without resizing or re‑encoding the image. If the image dimensions already match the grid alignment requirements, this optimization reduces token count computationally. It runs automatically before the Mini‑LM router is invoked, ensuring minimal overhead for images that are already optimally sized.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}