# How Image Compression Works in Headroom: A Technical Deep Dive

> Discover how Headroom slashes LLM token costs. Learn its technical approach to image compression, including Mini-LM classification and provider-specific optimization.

- Repository: [Tejas Chopra/headroom](https://github.com/chopratejas/headroom)
- Tags: deep-dive
- Published: 2026-06-14

---

**Headroom reduces LLM token costs by automatically detecting images in API requests, routing them through a trained Mini‑LM classifier, and applying provider‑specific compression techniques—such as OCR transcoding or resolution downsizing—before forwarding the request.**

The `chopratejas/headroom` repository implements an intelligent **image compression** pipeline that intercepts multimodal requests to minimize token usage. At its core, the `ImageCompressor` class orchestrates a zero‑loss tile optimization, a trained neural router, and provider‑specific transformations to reduce costs by 40–90% without sacrificing response quality.

## Detecting Images in the Message Payload

The pipeline begins in [`headroom/image/compressor.py`](https://github.com/chopratejas/headroom/blob/main/headroom/image/compressor.py) with the `has_images` method, which scans incoming message arrays for embedded images across OpenAI, Anthropic, and Google formats.

```python
def has_images(self, messages: list[dict[str, Any]]) -> bool:
    for message in messages:
        content = message.get("content")
        if isinstance(content, list):
            for item in content:
                if isinstance(item, dict):
                    # OpenAI format

                    if item.get("type") == "image_url": return True
                    # Anthropic format

                    if item.get("type") == "image": return True
                    # Google format

                    if "inlineData" in item: return True

```

*(see lines 74‑90 in [[`headroom/image/compressor.py`](https://github.com/chopratejas/headroom/blob/main/headroom/image/compressor.py)](https://github.com/chopratejas/headroom/blob/main/headroom/image/compressor.py#L74-L90))*

This detection handles the three provider‑specific image formats before any expensive processing occurs.

## Extracting Query Context and Image Bytes

Before routing decisions are made, Headroom extracts the textual prompt accompanying the image and retrieves the raw byte data for analysis.

### Extracting the User Query

The `_extract_query` method walks messages in reverse chronological order to find the most recent user text, which the router uses to determine intent.

```python
def _extract_query(self, messages):
    for message in reversed(messages):
        if message.get("role") != "user": continue
        content = message.get("content")
        if isinstance(content, str): return content
        if isinstance(content, list):
            texts = [item.get("text", "") for item in content
                     if isinstance(item, dict) and item.get("type") == "text"]
            if texts: return " ".join(texts)
    return ""

```

*(see lines 92‑117 in [[`headroom/image/compressor.py`](https://github.com/chopratejas/headroom/blob/main/headroom/image/compressor.py)](https://github.com/chopratejas/headroom/blob/main/headroom/image/compressor.py#L92-L117))*

### Pulling Raw Image Bytes

The `_extract_image_data` method normalizes the various provider formats (data URLs, base64 sources, inlineData) into raw bytes for downstream OCR and embedding analysis.

```python
def _extract_image_data(self, messages):
    for message in messages:
        content = message.get("content")
        if not isinstance(content, list): continue
        for item in content:
            if not isinstance(item, dict): continue
            # OpenAI – data URL

            if item.get("type") == "image_url" and (url := item.get("image_url", {}).get("url", "")).startswith("data:"):
                return base64.b64decode(re.match(r"data:image/[^;]+;base64,(.+)", url).group(1))
            # Anthropic – base64 source

            if item.get("type") == "image" and (src := item.get("source", {})).get("type") == "base64":
                return base64.b64decode(src.get("data", ""))
            # Google – inlineData

            if "inlineData" in item:
                return base64.b64decode(item["inlineData"].get("data", ""))
    return None

```

*(see lines 119‑149 in [[`headroom/image/compressor.py`](https://github.com/chopratejas/headroom/blob/main/headroom/image/compressor.py)](https://github.com/chopratejas/headroom/blob/main/headroom/image/compressor.py#L119-L149))*

## Pre‑Optimization: Tile‑Boundary Alignment

Before invoking the ML router, Headroom attempts a zero‑loss optimization by aligning image dimensions to the provider’s token grid. This can shave tokens without altering pixel data.

```python
from .tile_optimizer import optimize_images_in_messages
messages, tile_results = optimize_images_in_messages(messages, provider)

```

*(see lines 334‑339 in [[`headroom/image/compressor.py`](https://github.com/chopratejas/headroom/blob/main/headroom/image/compressor.py)](https://github.com/chopratejas/headroom/blob/main/headroom/image/compressor.py#L334-L339))*

If the image dimensions already line up with the provider’s token‑grid, the optimizer reduces token count computationally rather than visually.

## Routing with Mini‑LM and SigLIP

The compression strategy is determined by a **trained Mini‑LM classifier** (fine‑tuned on 1,157 examples) and an optional **SigLIP** model for image‑property analysis.

### Loading the Trained Router

The router is loaded lazily via `_get_router` to avoid unnecessary initialization overhead when processing text‑only requests.

```python
def _get_router(self) -> TrainedRouter:
    if self._router is None:
        from .trained_router import TrainedRouter
        self._router = TrainedRouter(
            model_path=self.model_id,
            use_siglip=self.use_siglip,
            device=self.device,
        )
    return self._router

```

*(see lines 154‑165 in [[`headroom/image/compressor.py`](https://github.com/chopratejas/headroom/blob/main/headroom/image/compressor.py)](https://github.com/chopratejas/headroom/blob/main/headroom/image/compressor.py#L154-L165))*

The `TrainedRouter` implementation lives in [`headroom/image/trained_router.py`](https://github.com/chopratejas/headroom/blob/main/headroom/image/trained_router.py) and encapsulates the classifier and SigLIP embedding logic.

### Classifying Query Intent and Image Signals

The router’s `classify` method first predicts a technique based on the textual query (`_classify_query`). If `use_siglip=True`, it extracts image embeddings and computes signal scores (`_analyze_image`) to adjust the final decision.

```python
technique, query_confidence = self._classify_query(query)
if self.use_siglip:
    image_embedding = self._get_image_embedding(image_data)
    image_signals = self._analyze_image(image_embedding)

# Apply heuristics → final RouteDecision

```

*(see the `classify` method in [[`headroom/image/trained_router.py`](https://github.com/chopratejas/headroom/blob/main/headroom/image/trained_router.py)](https://github.com/chopratejas/headroom/blob/main/headroom/image/trained_router.py#L334-L388))*

The router returns a `RouteDecision` containing:
- `technique` — one of `TRANSCODE`, `CROP`, `FULL_LOW`, or `PRESERVE`
- `confidence` — a probability‑like score indicating prediction certainty

## Applying Compression Techniques

The `_apply_compression` method in [`headroom/image/compressor.py`](https://github.com/chopratejas/headroom/blob/main/headroom/image/compressor.py) (lines 388‑460) transforms each image block based on the router’s decision.

### TRANSCODE: OCR Replacement

When the router selects `TRANSCODE`, Headroom runs OCR via `_ocr_extract` and substitutes the image block with extracted text.

```python
if technique.value == "transcode":
    extracted = self._ocr_extract(image_bytes_for_ocr)
    if extracted:
        new_content.append({"type": "text",
                            "text": f"[OCR from image]\n{extracted}"})

```

This eliminates image tokens entirely, replacing them with significantly cheaper text tokens.

### FULL_LOW and CROP: Resolution Reduction

For `FULL_LOW`, `CROP`, or when transcoding fails, Headroom applies provider‑specific resizing:

**OpenAI** — Sets `detail="low"` in the image URL:

```python
if provider == "openai":
    new_content.append({"type": "image_url",
                        "image_url": {..., "detail": "low"}})

```

**Anthropic** — Resizes to 512 px maximum dimension using Pillow’s `Image.resize`:

```python
elif provider == "anthropic":
    resized, media_type = self._resize_image(image_bytes_for_ocr, max_dimension=512)
    new_content.append({"type": "image",
                        "source": {"type": "base64",
                                   "media_type": media_type,
                                   "data": base64.b64encode(resized).decode()}})

```

**Google** — Resizes to 768 px maximum dimension:

```python
elif provider == "google":
    resized, media_type = self._resize_image(image_bytes_for_ocr, max_dimension=768)
    new_content.append({"inlineData": {"mimeType": media_type,
                                       "data": base64.b64encode(resized).decode()}})

```

*(see `_resize_image` at lines 251‑285 in [[`headroom/image/compressor.py`](https://github.com/chopratejas/headroom/blob/main/headroom/image/compressor.py)](https://github.com/chopratejas/headroom/blob/main/headroom/image/compressor.py#L251-L285))*

### PRESERVE: Pass‑Through

When confidence is low or the image requires full fidelity, the `PRESERVE` technique leaves the image untouched, forwarding the original bytes unchanged.

## Token Accounting and Savings Calculation

Headroom estimates token costs before and after compression using provider‑specific formulas. The results are stored in a `CompressionResult` object attached to the compressor instance.

```python
self.last_result = CompressionResult(
    technique=technique,
    original_tokens=original_tokens,
    compressed_tokens=compressed_tokens,
    confidence=confidence,
)

```

*(see lines 708‑714 in [[`headroom/image/compressor.py`](https://github.com/chopratejas/headroom/blob/main/headroom/image/compressor.py)](https://github.com/chopratejas/headroom/blob/main/headroom/image/compressor.py#L708-L714))*

The `last_savings` property calculates the percentage reduction, typically ranging from 40 % to 90 % depending on the technique applied.

## Usage Examples

### Basic Image Compression

Instantiate the `ImageCompressor` class directly to process message arrays before forwarding to an LLM provider:

```python
from headroom.image import ImageCompressor

compressor = ImageCompressor()
compressed = compressor.compress(messages, provider="openai")
print(f"Savings: {compressor.last_savings:.1f}%")
compressor.close()

```

### One‑Shot Helper Function

For simpler use cases, use the `compress_images` helper which handles initialization and cleanup:

```python
from headroom.image import compress_images

compressed = compress_images(messages, provider="anthropic")

```

*(see implementation at lines 734‑751 in [[`headroom/image/compressor.py`](https://github.com/chopratejas/headroom/blob/main/headroom/image/compressor.py)](https://github.com/chopratejas/headroom/blob/main/headroom/image/compressor.py#L734-L751))*

### Disabling Compression in Proxy Mode

When running the Headroom proxy server, disable image optimization entirely via CLI flag:

```bash
headroom proxy --no-image-optimize

```

This sets `proxy.config.image_optimize = False`, bypassing the `ImageCompressor` pipeline while preserving other proxy features.

## Summary

- **Headroom** intercepts multimodal LLM requests in `chopratejas/headroom` to reduce token costs through intelligent **image compression**.
- The **`ImageCompressor`** class detects images in OpenAI, Anthropic, and Google formats using `has_images` (lines 74‑90).
- A **Mini‑LM router** ([`trained_router.py`](https://github.com/chopratejas/headroom/blob/main/trained_router.py)) analyzes query text and optional SigLIP embeddings to select one of four techniques: `TRANSCODE`, `CROP`, `FULL_LOW`, or `PRESERVE`.
- **Zero‑loss tile optimization** (lines 334‑339) attempts cheap token savings before ML routing.
- **Provider‑specific resizing** uses Pillow’s `Image.resize` (lines 251‑285) to downscale images to 512 px (Anthropic) or 768 px (Google), while OpenAI requests use the `detail="low"` parameter.
- **OCR transcoding** replaces image bytes with extracted text blocks, eliminating image tokens entirely.
- Token savings are calculated and stored in `CompressionResult` (lines 708‑714), with typical reductions of 40–90 %.

## Frequently Asked Questions

### What compression techniques does Headroom support?

Headroom supports four techniques defined in the `Technique` enum: **`TRANSCODE`** (OCR text replacement), **`CROP`** (smart region selection), **`FULL_LOW`** (resolution reduction), and **`PRESERVE`** (original image). The `TRANSCODE` technique eliminates image tokens entirely by substituting OCR‑extracted text, while `FULL_LOW` uses provider‑specific resizing (512 px for Anthropic, 768 px for Google, or `detail="low"` for OpenAI).

### How does Headroom decide which compression technique to use?

A **trained Mini‑LM classifier** (`TrainedRouter` in [`headroom/image/trained_router.py`](https://github.com/chopratejas/headroom/blob/main/headroom/image/trained_router.py)) analyzes the user’s textual query to predict the appropriate technique. If the `use_siglip` option is enabled, the router also extracts image embeddings using SigLIP to compute signal scores (e.g., text density, visual complexity) that can override or adjust the initial prediction. The final `RouteDecision` includes both the selected technique and a confidence score.

### Can I use Headroom image compression without the proxy server?

Yes. The **`ImageCompressor`** class and the **`compress_images`** helper function are designed for standalone use. Import them from `headroom.image` and call `compressor.compress(messages, provider="...")` before forwarding the modified payload to your LLM provider. This integration pattern works with direct API calls, serverless functions, or custom middleware without requiring the Headroom proxy.

### What is the tile‑boundary optimization and when does it apply?

The **tile‑boundary optimization** is a zero‑loss preprocessing step (implemented in [`headroom/image/tile_optimizer.py`](https://github.com/chopratejas/headroom/blob/main/headroom/image/tile_optimizer.py)) that aligns image dimensions to the provider’s token grid without resizing or re‑encoding the image. If the image dimensions already match the grid alignment requirements, this optimization reduces token count computationally. It runs automatically before the Mini‑LM router is invoked, ensuring minimal overhead for images that are already optimally sized.