Troubleshooting Guide: What to Do When Headroom Compression Isn't Reducing Tokens Sufficiently

When Headroom compression fails to reduce token counts sufficiently, the issue typically stems from conservative threshold settings in CompressionPolicy or transformer-specific min_tokens configurations that prevent the compression pipelines from firing.

When Headroom compression isn't reducing tokens sufficiently, you need to examine the interaction between authentication modes and transformer thresholds governing the compression pipelines. The Headroom library processes prompts through a series of specialized transforms—including SmartCrusher for JSON arrays and LogCompressor for log files—but these only activate when specific token count and compression ratio thresholds are met. Understanding how to adjust these levers in headroom/transforms/compression_policy.py and the individual transformer configs will restore meaningful token reduction.

Common Symptoms and Diagnostic Patterns

When token reduction falls short, the symptoms point to specific configuration gates. Use this diagnostic matrix to identify which threshold is blocking compression:

Symptom	Likely Cause	Fix
Only a tiny token drop	Transformer never fires because input is below min-token thresholds	Lower min_tokens_to_crush in SmartCrusherConfig
Compression ratio stays ≈ 1.0	CompressionPolicy disables lossy compression (max_lossy_ratio too low) or forces read-only TOIN	Use policy_for_mode(AuthMode.PAYG) or raise _MAX_LOSSY_RATIO_PAYG
CCR markers never appear	min_compression_ratio_for_ccr threshold not met	Decrease the in-code constant or raise max_lossy_ratio
Large JSON arrays stay untouched	Lossless-first path wins only when it saves ≥ 30% (lossless_min_savings_ratio)	Reduce SmartCrusherConfig.lossless_min_savings_ratio or force lossy path
Log files give no reduction	min_lines_for_ccr not met or ratio exceeds threshold	Lower LogCompressorConfig.min_lines_for_ccr or adjust ratio threshold

Key Architectural Levers to Adjust

Understanding these four architectural components helps you pinpoint where to apply pressure when Headroom compression isn't reducing tokens sufficiently.

CompressionPolicy and Authentication Modes

The CompressionPolicy gates aggressive compression based on the active AuthMode. Key constants like _VOLATILE_TOKEN_THRESHOLD_PAYG and _MAX_LOSSY_RATIO_PAYG in headroom/transforms/compression_policy.py determine whether lossy algorithms and cache-alignment features are enabled. Subscription mode defaults to conservative settings that may disable TOIN writes entirely.

Transformer Configuration Dataclasses

Each compressor exposes a tunable dataclass with hard thresholds:

• SmartCrusherConfig – controls min_tokens_to_crush, lossless_min_savings_ratio, and max_items_after_crush
• LogCompressorConfig – defines min_lines_for_ccr and references the inline constant min_compression_ratio_for_ccr (default 0.5)

These reside in headroom/transforms/smart_crusher.py and headroom/transforms/log_compressor.py respectively.

Cache-Compression-Retrieval (CCR) Markers

CCR markers (<<ccr:) indicate when the lossy path has dropped rows and the data can be retrieved on demand. These markers only appear when the compression ratio falls below hard-coded thresholds—0.8 for search results and 0.5 for logs. If markers are absent, compression occurred but did not meet the ratio requirements for caching.

Tokenizer Implementation

All transforms rely on the Tokenizer class in headroom/tokenizer.py to decide if input is "large enough" to merit compression. A misconfigured tokenizer that under-counts tokens can hide compression opportunities by making inputs appear smaller than they actually are.

Step-by-Step Troubleshooting Workflow

Follow this sequence to diagnose why Headroom compression isn't reducing tokens sufficiently:

1. Inspect the applied policy – Verify the request isn't falling back to the read-only Subscription policy:

from headroom.transforms.compression_policy import resolve_policy, AuthMode

policy = resolve_policy(AuthMode.PAYG)   # or AuthMode.OAUTH for aggressive mode

print(policy)   # should show live_zone_only=False, cache_aligner_enabled=True

2. Verify token thresholds – Print the configured limits for your target transformer:

from headroom.transforms.smart_crusher import SmartCrusherConfig

cfg = SmartCrusherConfig()
print("min_tokens_to_crush:", cfg.min_tokens_to_crush)
print("lossless_min_savings_ratio:", cfg.lossless_min_savings_ratio)

3. Force a more aggressive run – Temporarily override defaults to test if token count improves:

from headroom.transforms.smart_crusher import smart_crush_tool_output, SmartCrusherConfig

cfg = SmartCrusherConfig(min_tokens_to_crush=50, lossless_min_savings_ratio=0.05)
crushed, was_modified, info = smart_crush_tool_output(content, config=cfg)
print("crushed tokens:", len(crushed))

4. Check CCR marker creation – Look for <<ccr: markers in output. Absence indicates the ratio threshold was not met:

if "<<ccr:" not in crushed:
    print("CCR marker missing – consider lowering min_compression_ratio_for_ccr")

5. Adjust the policy globally – Edit per-mode defaults in headroom/transforms/compression_policy.py to increase allowed loss:

# In compression_policy.py

_MAX_LOSSY_RATIO_PAYG = 0.6   # increase from 0.45 to allow more loss

After changes, run pytest -q tests/test_compression_policy.py to confirm parity with the Rust reference.

6. Run the full pipeline with diagnostics – The ContentRouter aggregates results from all transforms:

from headroom.transforms.content_router import ContentRouter, ContentRouterConfig

router = ContentRouter(ContentRouterConfig(enable_tabular_compressor=True))
result = router.compress(long_prompt)
print("overall compression_ratio:", result.compression_ratio)
for log in result.routing_log:
    print(log.transform_name, log.compression_ratio)

The routing_log entries pinpoint which transform contributed the most reduction.

Practical Configuration Examples

Diagnose the Active CompressionPolicy

Determine which mode-specific thresholds are currently active:

from headroom.transforms.compression_policy import resolve_policy, AuthMode

def show_policy(auth):
    pol = resolve_policy(auth)
    print(f"Policy for {auth.name}:")
    print(f"  live_zone_only = {pol.live_zone_only}")
    print(f"  cache_aligner_enabled = {pol.cache_aligner_enabled}")
    print(f"  volatile_token_threshold = {pol.volatile_token_threshold}")
    print(f"  max_lossy_ratio = {pol.max_lossy_ratio}")
    print(f"  toin_read_only = {pol.toin_read_only}")

show_policy(AuthMode.PAYG)          # aggressive mode

show_policy(AuthMode.SUBSCRIPTION)  # conservative mode

Lower SmartCrusher Token Thresholds

Reduce the minimum token count and lossless savings requirement to force compression on smaller inputs:

from headroom.transforms.smart_crusher import smart_crush_tool_output, SmartCrusherConfig

# Original: min_tokens_to_crush = 200

cfg = SmartCrusherConfig(min_tokens_to_crush=80, lossless_min_savings_ratio=0.05)
crushed, modified, info = smart_crush_tool_output(my_tool_output, config=cfg)

print("Modified:", modified)
print("Info tag:", info)
print("Result length:", len(crushed))

Force CCR Markers for Log Compression

Override the default line count requirements to ensure CCR markers appear:

from headroom.transforms.log_compressor import LogCompressor, LogCompressorConfig

cfg = LogCompressorConfig(min_lines_for_ccr=2, enable_ccr=True)
compressor = LogCompressor(cfg)
result = compressor.compress(log_text)

if "<<ccr:" in result.compressed:
    print("CCR marker present – will be retrieved on demand.")
else:
    print("No CCR marker – compression ratio may be too high.")

Inspect Per-Transform Compression Ratios

Identify which stage in the pipeline is underperforming:

from headroom.transforms.content_router import ContentRouter, ContentRouterConfig

router = ContentRouter(ContentRouterConfig())
out = router.compress(large_prompt)

print("Overall ratio:", out.compression_ratio)
for step in out.routing_log:
    print(f"{step.transform_name} → {step.compression_ratio:.2f}")

Summary

When Headroom compression isn't reducing tokens sufficiently:

• Check the active CompressionPolicy – Ensure you're using AuthMode.PAYG or AuthMode.OAUTH rather than the conservative Subscription mode that disables lossy compression.
• Lower transformer-specific thresholds – Reduce min_tokens_to_crush in SmartCrusherConfig and min_lines_for_ccr in LogCompressorConfig to trigger compression on smaller inputs.
• Adjust lossy ratio limits – Increase _MAX_LOSSY_RATIO_PAYG or reduce lossless_min_savings_ratio to allow the lossy compression path to win.
• Verify CCR marker emission – Confirm that compression ratios meet the hard-coded thresholds (0.8 for search, 0.5 for logs) required for cache-compression-retrieval markers.
• Inspect the ContentRouter logs – Use the routing_log to identify which specific transform is failing to reduce token count.

Frequently Asked Questions

Why does Headroom compression show a ratio of 1.0 with no token reduction?

A compression ratio of 1.0 indicates that the CompressionPolicy is either disabling lossy compression or the input falls below the min_tokens_to_crush threshold. Check that you're using AuthMode.PAYG rather than AuthMode.SUBSCRIPTION, as the latter sets toin_read_only=True and disables aggressive compression. Also verify that max_lossy_ratio in your policy is set high enough to allow row dropping.

How do I force aggressive compression on small JSON arrays?

Small inputs often bypass compression because they don't meet the min_tokens_to_crush default (typically 200 tokens). Create a custom SmartCrusherConfig with min_tokens_to_crush=50 and lossless_min_savings_ratio=0.05 to force the compressor to evaluate smaller arrays. Alternatively, set with_compaction=False to bypass the lossless-first path that requires 30% savings to trigger.

What do CCR markers indicate and why are they missing?

CCR (Cache-Compression-Retrieval) markers like <<ccr: indicate that data was compressed via the lossy path and can be retrieved on demand. They appear only when the compression ratio falls below specific thresholds—0.8 for search results in headroom/transforms/search_compressor.py and 0.5 for logs in headroom/transforms/log_compressor.py. If markers are missing, your compression ratio is not meeting these hard-coded limits; lower min_compression_ratio_for_ccr or increase the allowed loss ratio to enable them.

Where are the compression thresholds actually enforced?

Token count thresholds are enforced in the individual transformer configs (SmartCrusherConfig, LogCompressorConfig), while policy-level gating occurs in headroom/transforms/compression_policy.py. The ContentRouter orchestrates the pipeline but respects the min_tokens and compression ratio checks implemented in each specific transform module. All token counting uses the Tokenizer implementation defined in headroom/tokenizer.py.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}