How to Troubleshoot Accuracy Reduction in Headroom Compression: A Complete Guide

To troubleshoot accuracy reduction in Headroom compression, enable debug logging to inspect Smart Crusher transforms, adjust the SmartCrusherConfig thresholds and compaction formats, and verify CCR marker retrieval when rows are dropped.

When using the chopratejas/headroom library for LLM context-window optimization, aggressive compression can inadvertently strip critical information from tool outputs. This guide explains how to diagnose and fix accuracy loss caused by the Smart Crusher transform, which is implemented in Rust and exposed through headroom/transforms/smart_crusher.py.

Understanding Why Compression Reduces Accuracy

The Smart Crusher reduces token usage through lossy and lossless strategies. When misconfigured, these strategies can remove data that the LLM needs to generate correct answers.

Row Drop and CCR Sentinels

The Content Compression Retrieval (CCR) sentinel removes rows deemed low-value and inserts a {"_ccr_dropped": "..."} marker. The LLM sees this marker but not the original rows unless it explicitly requests them via the CCR retrieval tool.

In SmartCrusher._smart_crush_content() (lines 66-78), the transform decides which rows to drop. If the model never requests the marker, the information is permanently lost, causing factual gaps in responses.

Lossless Compaction Format Issues

JSON arrays are compacted into formats like CSV-schema or markdown-kv. The compaction_format validation occurs in SmartCrusher.__init__() against _SUPPORTED_COMPACTION_FORMATS (lines 60-64).

If you select "markdown-kv", column names may be discarded. Without field names, the model can misinterpret data values, leading to classification errors or incorrect reasoning.

Threshold and Limit Misconfiguration

The compressor only activates when outputs exceed min_tokens_to_crush (default 200) and after seeing min_items_to_analyze (default 5). Additionally, max_items_after_crush defaults to 15.

When SmartCrusherConfig thresholds are too low (lines 67-71), the system compresses small, information-dense payloads. When max_items_after_crush (lines 73-74) is too restrictive, answer-relevant rows get discarded from large datasets.

Identifying Accuracy Issues

Watch for these specific symptoms that indicate compression is harming model performance:

• Missing items in the LLM's answer that exist in the original tool output
• Presence of <<ccr:...>> markers in prompts that the model never expands
• Accuracy drops on benchmark suites like GSM8K or TruthfulQA after enabling compression

Step-by-Step Troubleshooting Guide

Enable Debug Logging

Start by inspecting which transforms are applied to your tool outputs. Configure Python's logging module to capture Headroom's internal operations:

import logging
logging.basicConfig(level=logging.DEBUG)
logging.getLogger("headroom").setLevel(logging.DEBUG)

Look for log entries containing smart:smart_crush:... and CCR marker insertions to confirm the transform is running.

Inspect Transform Metadata

Use the simulation API to preview exactly what the compressor will do before sending a real request:

plan = client.chat.completions.simulate(
    model="gpt-4o",
    messages=messages,
)
print(plan.transforms)          # e.g., ["smart:smart_crush:3:search"]

print(plan.tokens_before, plan.tokens_after)

The plan.transforms list contains tags using the format smart_crush:<count>[:<tool names>] defined in smart_crusher.py (lines 29-38). Compare tokens_before and tokens_after to see if compression is excessive.

Adjust SmartCrusher Configuration

Tune the SmartCrusherConfig to make compression less aggressive. These fields are defined in lines 67-80 of smart_crusher.py.

Raise the token threshold to exempt small payloads:

from headroom import HeadroomConfig, SmartCrusherConfig
cfg = HeadroomConfig()
cfg.smart_crusher.min_tokens_to_crush = 500   # default is 200

client = HeadroomClient(..., config=cfg)

Increase retention limits to keep more rows:

cfg.smart_crusher.max_items_after_crush = 50   # default is 15

Select a safer compaction format that preserves schema information:

cfg.smart_crusher.compaction_format = "csv-schema"  # instead of "markdown-kv"

Disable the transform entirely to confirm it's the accuracy source:

cfg.smart_crusher.enabled = False

Skip Compression for Specific Tools

When only certain tools produce problematic outputs, use the per-request headroom_tool_profiles API to disable compression selectively:

response = client.chat.completions.create(
    model="gpt-4o",
    messages=messages,
    headroom_tool_profiles={
        "search": {"skip_compression": True},   # preserve original search results

    },
)

Verify CCR Retrieval

If prompts contain <<ccr:…>> markers but the model reports missing data, verify the retrieval mechanism is accessible. You can manually fetch dropped content from the compression store:

# Retrieve original rows using the hash from the marker

original = client.compression_store.get(hash="abc123")
print(original)   # contains the full dropped rows

The bridge that mirrors Rust-stored CCR entries into Python lives in smart_crusher.py methods _mirror_ccr_to_python_store and _mirror_single_hash_to_python_store (lines 84-111).

Benchmark Accuracy Changes

Quantify the impact of configuration changes using the built-in benchmark suite:

python -m benchmarks.compression_benchmark --suite gsm8k

Compare the accuracy column before and after adjustments. Higher thresholds and retention limits should correlate with improved accuracy scores.

Code Examples for Configuration

Here is a complete configuration example that implements conservative compression settings:

# Enable verbose logging for diagnostics

import logging
logging.basicConfig(level=logging.DEBUG)
logging.getLogger("headroom").setLevel(logging.DEBUG)

# Configure conservative compression limits

from headroom import HeadroomConfig, HeadroomClient
from openai import OpenAI

cfg = HeadroomConfig()
cfg.smart_crusher.min_tokens_to_crush = 500      # only compress large payloads

cfg.smart_crusher.max_items_after_crush = 50   # retain more rows

cfg.smart_crusher.compaction_format = "csv-schema"  # preserve column names

client = HeadroomClient(original_client=OpenAI(), config=cfg)

# Simulate to verify transforms before sending

plan = client.chat.completions.simulate(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Analyze the data"}],
)
print(f"Transforms: {plan.transforms}")
print(f"Token change: {plan.tokens_before} → {plan.tokens_after}")

# Skip compression for critical tools

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Search and analyze"}],
    headroom_tool_profiles={
        "database_query": {"skip_compression": True}
    },
)

# Manual CCR retrieval if needed

if "ccr" in str(response):
    hash_key = "a1b2c3d4e5f6"  # extract from marker

    original_rows = client.compression_store.get(hash=hash_key)
    print(f"Retrieved {len(original_rows)} dropped rows")

Summary

• Debug first: Enable logging and use client.chat.completions.simulate() to inspect the smart_crush transform metadata before production use.
• Configure thresholds: Raise min_tokens_to_crush and min_items_to_analyze in SmartCrusherConfig to prevent compressing small, dense payloads.
• Preserve data: Increase max_items_after_crush beyond the default 15 and use "csv-schema" instead of "markdown-kv" to retain column names.
• Selective compression: Use headroom_tool_profiles with skip_compression for tools that provide critical answer data.
• Verify retrieval: Ensure the model can access CCR-stored data via client.compression_store.get() when _ccr_dropped markers appear.

Frequently Asked Questions

Why does the LLM miss information that exists in the original tool output?

The Smart Crusher likely dropped the rows using the CCR sentinel and inserted a {"_ccr_dropped": "..."} marker. If the LLM never calls the CCR retrieval tool to expand the marker, it never sees the original data. Check smart_crusher.py lines 66-78 to see how the _smart_crush_content() method decides which rows to drop.

How can I tell if compression is affecting my specific request?

Call client.chat.completions.simulate() before the actual request. Inspect the plan.transforms list for smart_crush entries and compare tokens_before and tokens_after. If the token reduction is drastic (e.g., 80%+) and you see <<ccr:...>> markers, the content is being aggressively compressed.

What is the safest compaction format to preserve accuracy?

Use "csv-schema" instead of "markdown-kv". The "csv-schema" format preserves column headers and structure, whereas "markdown-kv" can discard field names that the LLM needs for correct interpretation. This is validated in SmartCrusher.__init__() against _SUPPORTED_COMPACTION_FORMATS in smart_crusher.py.

How do I recover data from a CCR marker if the model doesn't retrieve it?

Access the compression_store directly using the hash from the marker:

data = client.compression_store.get(hash="marker_hash_here")

The store is populated by _mirror_ccr_to_python_store and _mirror_single_hash_to_python_store in smart_crusher.py (lines 84-111), which bridge the Rust compression backend to Python.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}