# Debugging Headroom: What to Do When Compression Breaks Agent Behavior

> Fix agent behavior issues caused by Headroom compression. Learn to identify and resolve misrouted payloads, compression layer timeouts, and metadata persistence failures.

- Repository: [Tejas Chopra/headroom](https://github.com/chopratejas/headroom)
- Tags: troubleshooting
- Published: 2026-06-20

---

**When Headroom compression breaks agent behavior, the root cause is typically a misrouted payload in the Content Router, a timeout or exception in the Rust-backed compression layer, or a failure to persist metadata in the Compression Store (CCR) that forces the pipeline to return uncompressed text.**

Headroom is an open-source token-compression framework designed to reduce LLM context-window usage through content-specific transforms. When the compression pipeline malfunctions, downstream agents may exhibit garbled output, context loss, or complete stalls. Debugging these issues requires tracing the execution flow through [[`headroom/transforms/pipeline.py`](https://github.com/chopratejas/headroom/blob/main/headroom/transforms/pipeline.py)](https://github.com/chopratejas/headroom/blob/main/headroom/transforms/pipeline.py) and verifying the health of each architectural layer.

## How Headroom's Compression Pipeline Works

The Headroom architecture routes raw text through a series of specialized compressors before the final payload reaches the agent. Understanding these layers is essential for identifying where the breakage occurs.

### Content Router

The **Content Router** residing in [[`headroom/transforms/pipeline.py`](https://github.com/chopratejas/headroom/blob/main/headroom/transforms/pipeline.py)](https://github.com/chopratejas/headroom/blob/main/headroom/transforms/pipeline.py) inspects incoming content and selects the appropriate compressor based on payload type—code, logs, search results, or tabular data. Mis-routing (for example, sending code to the log compressor) leads to unexpected truncation or token explosions.

### Transform-Specific Compressors

Each route invokes a dedicated compressor implemented in its own module:

- **SmartCrusher** ([[`headroom/transforms/smart_crusher.py`](https://github.com/chopratejas/headroom/blob/main/headroom/transforms/smart_crusher.py)](https://github.com/chopratejas/headroom/blob/main/headroom/transforms/smart_crusher.py)): AST-based compression for source code using Tree-sitter.
- **LogCompressor** ([[`headroom/transforms/log_compressor.py`](https://github.com/chopratejas/headroom/blob/main/headroom/transforms/log_compressor.py)](https://github.com/chopratejas/headroom/blob/main/headroom/transforms/log_compressor.py)): Pattern-based reduction for log lines.
- **SearchCompressor** ([[`headroom/transforms/search_compressor.py`](https://github.com/chopratejas/headroom/blob/main/headroom/transforms/search_compressor.py)](https://github.com/chopratejas/headroom/blob/main/headroom/transforms/search_compressor.py)): Snippet summarization for search results.
- **Kompress** ([[`headroom/transforms/kompress_compressor.py`](https://github.com/chopratejas/headroom/blob/main/headroom/transforms/kompress_compressor.py)](https://github.com/chopratejas/headroom/blob/main/headroom/transforms/kompress_compressor.py)): Rust-backed neural compression requiring native binaries in `headroom/_core`.

### Compression Store (CCR)

The **Compression Cache and Repository (CCR)** in [[`headroom/cache/compression_store.py`](https://github.com/chopratejas/headroom/blob/main/headroom/cache/compression_store.py)](https://github.com/chopratejas/headroom/blob/main/headroom/cache/compression_store.py) persists compression metadata for learning and caching. When the store is unavailable, the pipeline falls back to passthrough mode, returning the original text and breaking agents that expect compressed input.

### Observability Layer

Metrics emission is handled by [[`headroom/transforms/observability.py`](https://github.com/chopratejas/headroom/blob/main/headroom/transforms/observability.py)](https://github.com/chopratejas/headroom/blob/main/headroom/transforms/observability.py). Missing observability signals can mask silent failures, making it appear as though compression succeeded when it actually aborted.

## Common Failure Triggers That Disrupt Agents

### Missing Tree-sitter Dependencies

**SmartCrusher** relies on the `tree_sitter` Python package to parse source code. If Tree-sitter cannot be imported, the compressor falls back to a no-op, but the router still expects a shortened result. This mismatch causes downstream agents to receive the original, oversized text.

### Compression Store Errors

The store may raise exceptions (for example, database connection loss) inside the `get_compression_store()` function (see lines 795–818 in [[`headroom/transforms/smart_crusher.py`](https://github.com/chopratejas/headroom/blob/main/headroom/transforms/smart_crusher.py)](https://github.com/chopratejas/headroom/blob/main/headroom/transforms/smart_crusher.py)). While the code catches and logs these errors, a complete store outage forces the pipeline to skip the `record_compression` step, leading to repeated re-compression of the same payload.

### Threshold Misconfiguration

Every compressor defines a `min_compression_ratio_for_ccr` threshold (for example, `0.5` for logs, `0.8` for search). If the actual compression ratio exceeds this value, the result is treated as **passthrough**. Agents relying on a compressed payload then receive the full original text, blowing token budgets.

### Pipeline Timeouts

The environment variable `HEADROOM_COMPRESSION_TIMEOUT_SECONDS` defaults to 30 seconds. Long-running Rust models (Kompress) that exceed this limit abort and return the original text, causing agents to process unreduced context.

### Tag Protector Interference

The **Tag Protector** in [[`headroom/transforms/tag_protector.py`](https://github.com/chopratejas/headroom/blob/main/headroom/transforms/tag_protector.py)](https://github.com/chopratejas/headroom/blob/main/headroom/transforms/tag_protector.py) shields custom XML tags from compression via `protect_tags` and `restore_tags`. Incorrect boundary definitions can cause the protector to strip essential markers, leaving the router without valid compression hints and breaking agent parsers.

## Step-by-Step Debugging Workflow

1. **Validate Router Decisions**
   Inspect debug logs for messages like “ContentRouter: routing to `log_compressor`”. Verify the selected compressor matches the payload type—code should route to **SmartCrusher**, logs to **LogCompressor**.

2. **Check Compressor Health**
   - Search logs for `compression_ratio`. A value of `1.0` indicates no compression occurred.
   - Confirm the Rust binary is loadable by checking for warnings such as “Kompress model X not cached” in `headroom/_core`.

3. **Confirm Store Availability**
   Look for the log line “CCR mirror: compression_store module unavailable”. If present, verify that the optional `headroom-cache` package is installed and that `HEADROOM_CACHE_URL` (if used) is correctly configured.

4. **Review Threshold Settings**
   Open the compressor config (for example, `LogCompressorConfig` in [[`headroom/transforms/log_compressor.py`](https://github.com/chopratejas/headroom/blob/main/headroom/transforms/log_compressor.py)](https://github.com/chopratejas/headroom/blob/main/headroom/transforms/log_compressor.py)). Lower `min_compression_ratio_for_ccr` temporarily to force compression events and surface hidden paths.

5. **Enable Detailed Observability**
   Instantiate a `PrometheusCompressionObserver` (see [[`headroom/transforms/observability.py`](https://github.com/chopratejas/headroom/blob/main/headroom/transforms/observability.py)](https://github.com/chopratejas/headroom/blob/main/headroom/transforms/observability.py)) and call `record_compression` after each transform. A sudden drop to zero events indicates a silent failure.

6. **Reproduce with Unit Tests**
   Run the relevant test suites to verify compressor behavior:

   ```bash
   pytest tests/test_transforms_log_compressor.py::test_log_compression_ratio
   pytest tests/test_transforms_search_compressor.py::test_search_compression_result_properties
   ```

   These tests assert that `result.compression_ratio` stays below the configured threshold and that the store interaction succeeds.

7. **Inspect Tag Boundaries**
   Verify that custom tags are correctly escaped as `<mytag>…</mytag>`. Ensure `protect_tags` wraps content without stripping inner text, which would confuse the downstream agent parser.

## Configuration Fixes and Validation Code

### Forcing the Log Compressor to Run

Use this snippet to bypass the compression ratio threshold and verify the compressor logic itself:

```python
from headroom.transforms.log_compressor import LogCompressor, LogCompressorConfig

cfg = LogCompressorConfig(min_compression_ratio_for_ccr=0.0)  # disable threshold

compressor = LogCompressor(cfg)

payload = "2023-01-01 12:00:00 INFO Starting service...\n" * 20
result = compressor.apply(payload, compression_policy=None)

print("Compressed length:", len(result.compressed))
print("Ratio:", result.compression_ratio)

```

*Relevant source:* [[`headroom/transforms/log_compressor.py`](https://github.com/chopratejas/headroom/blob/main/headroom/transforms/log_compressor.py)](https://github.com/chopratejas/headroom/blob/main/headroom/transforms/log_compressor.py)

### Recording a Compression Event Manually

Verify that observability hooks are correctly wired by emitting a manual metric:

```python
from headroom.transforms.observability import PrometheusCompressionObserver
from headroom.transforms.smart_crusher import SmartCrusher

observer = PrometheusCompressionObserver()
crusher = SmartCrusher()
payload = "def foo():\n    return 42\n"
result = crusher.apply(payload, compression_policy=None)

observer.record_compression(
    strategy="smart_crusher",
    compression_ratio=result.compression_ratio,
    original_len=len(payload),
    compressed_len=len(result.compressed),
)

```

*Relevant source:* [[`headroom/transforms/observability.py`](https://github.com/chopratejas/headroom/blob/main/headroom/transforms/observability.py)](https://github.com/chopratejas/headroom/blob/main/headroom/transforms/observability.py)

### Simulating a Store Failure

Test graceful degradation when the CCR store is unreachable:

```python
from unittest.mock import patch
from headroom.transforms.smart_crusher import SmartCrusher

def broken_store(*_, **__):
    raise RuntimeError("Simulated store outage")

with patch("headroom.transforms.smart_crusher.get_compression_store", broken_store):
    crusher = SmartCrusher()
    # This will still compress but skip storing the metadata

    result = crusher.apply("some long text …")
    print("Compression succeeded; store was bypassed.")

```

*Relevant source:* Lines 795–819 in [[`headroom/transforms/smart_crusher.py`](https://github.com/chopratejas/headroom/blob/main/headroom/transforms/smart_crusher.py)](https://github.com/chopratejas/headroom/blob/main/headroom/transforms/smart_crusher.py)

## Summary

- **Headroom** routes content through specialized compressors in [[`headroom/transforms/pipeline.py`](https://github.com/chopratejas/headroom/blob/main/headroom/transforms/pipeline.py)](https://github.com/chopratejas/headroom/blob/main/headroom/transforms/pipeline.py); mis-routing is the most common cause of broken agent behavior.
- **Tree-sitter** and **Rust binaries** (`headroom/_core`) are hard dependencies for code and neural compression; missing them triggers silent no-op fallbacks.
- **Thresholds** defined by `min_compression_ratio_for_ccr` can cause the pipeline to return uncompressed text if the ratio is too high.
- **Timeouts** governed by `HEADROOM_COMPRESSION_TIMEOUT_SECONDS` abort slow compressors, returning the original payload.
- **CCR store** failures (monitored via `get_compression_store()`) disable metadata persistence but should not block the compression itself.
- Use **unit tests** from [`tests/test_transforms_log_compressor.py`](https://github.com/chopratejas/headroom/blob/main/tests/test_transforms_log_compressor.py) and [`tests/test_transforms_search_compressor.py`](https://github.com/chopratejas/headroom/blob/main/tests/test_transforms_search_compressor.py) to validate fixes.

## Frequently Asked Questions

### Why is my agent receiving uncompressed text even though Headroom is enabled?

The **Content Router** may have selected a passthrough route due to a high `min_compression_ratio_for_ccr` threshold, or the **Compression Store** may be unavailable, forcing a fallback. Check debug logs for `compression_ratio: 1.0` and verify that `HEADROOM_COMPRESSION_TIMEOUT_SECONDS` has not been exceeded.

### How do I identify which compressor Headroom selected for my payload?

Enable debug logging by setting `export HEADROOM_LOG_LEVEL=debug`, then look for entries emitted by [[`headroom/transforms/pipeline.py`](https://github.com/chopratejas/headroom/blob/main/headroom/transforms/pipeline.py)](https://github.com/chopratejas/headroom/blob/main/headroom/transforms/pipeline.py) containing the string “ContentRouter: routing to”. The log line will specify the compressor class name (for example, `smart_crusher` or `log_compressor`).

### What causes the "compression_store module unavailable" warning in logs?

This warning appears when the CCR import in [[`headroom/cache/compression_store.py`](https://github.com/chopratejas/headroom/blob/main/headroom/cache/compression_store.py)](https://github.com/chopratejas/headroom/blob/main/headroom/cache/compression_store.py) fails, usually because the optional `headroom-cache` dependency is not installed or because `HEADROOM_CACHE_URL` points to an unreachable endpoint. Install the extra dependency or correct the URL to resolve the warning.

### How can I adjust compression thresholds to force compression during testing?

Lower the `min_compression_ratio_for_ccr` value in the compressor’s configuration class (for example, `LogCompressorConfig` or the equivalent in `SmartCrusher`). Setting it to `0.0` disables the threshold entirely, ensuring the compressor always runs and allowing you to verify agent behavior against compressed output regardless of the achieved ratio.