Headroom's Kompress Compressor: ModernBERT Token Classification for LLM Context

Kompress is Headroom's built-in ML compressor that uses a dual-head ModernBERT model to classify tokens as keep or discard, achieving 80-95% compression while preserving semantic meaning for downstream LLMs.

The chopratejas/headroom repository includes a sophisticated token classification system called Kompress that reduces context window usage by intelligently pruning unnecessary tokens from tool outputs and assistant messages. This compressor leverages a custom dual-head ModernBERT architecture to perform binary token classification and span importance scoring, enabling aggressive compression without sacrificing the contextual details required by large language models.

How Kompress Works: Dual-Head ModernBERT Architecture

In headroom/transforms/kompress_compressor.py, the HeadroomCompressorModel implements a specialized architecture built on ModernBERT, a BERT-style encoder optimized for token-level relevance tasks rather than standard masked language modeling.

The model features two complementary classification heads:

Token-Level Binary Classification

The self.token_head applies a binary linear classification to each input token, predicting whether individual tokens should be retained or discarded. This head operates on the hidden states produced by the ModernBERT encoder base.

Span Importance Scoring

A lightweight 1-D convolutional neural network (self.span_conv) performs span-importance scoring to identify semantically coherent regions like code blocks or file paths. This prevents the compressor from fragmenting important contextual spans during the pruning process.

The final keep mask is computed as the logical OR of the token head's decision and a "borderline-token-boosted-by-span" rule (see lines 31-33 of the model definition), ensuring that tokens within important spans receive preservation priority.

The Three-Stage Compression Pipeline

Kompress operates through a distinct inference pipeline implemented in the _compress method:

1. Model Loading and Initialization

On first use, the system auto-downloads model weights from chopratejas/kompress-v2-base and tokenizer configuration from answerdotai/ModernBERT-base. The loading logic in _load_kompress (lines 78-104) supports multiple backends:

• ONNX: Lightweight CPU inference
• PyTorch: GPU or Apple Silicon (MPS) acceleration
• Disable: Passthrough mode via HEADROOM_KOMPRESS_BACKEND=disable

2. Chunked Inference

Input text is split into 350-word chunks. Each chunk undergoes tokenization and forward pass through the model. The get_keep_mask method (for ONNX) or get_scores method (for PyTorch) generates binary masks or probability scores. The PyTorch implementation combines token-head probabilities with span convolutions to determine final retention decisions.

3. Re-assembly and CCR Caching

Kept token indices merge back into the compressed string. If the compression ratio falls below the CCR threshold (approximately 0.8), the result is stored via _store_in_ccr (lines 140-150) with a hash-based cache key appended to the output for later retrieval.

Installation and Backend Configuration

Kompress operates as an opt-in ML compression feature. Install the optional dependencies:

pip install "headroom-ai[ml]"

Control the inference backend using environment variables:

import os
os.environ["HEADROOM_KOMPRESS_BACKEND"] = "onnx"  # or "pytorch", "disable"

When running on GPU hardware, Kompress supports batch inference for multiple texts in a single forward pass, providing approximately 2× speedup for three or more concurrent items.

Source Code Implementation Details

The core implementation resides in headroom/transforms/kompress_compressor.py, which defines:

• KompressConfig: Runtime configuration dataclass
• KompressResult: Compression metrics including tokens_saved and savings_percentage
• KompressCompressor: Main interface with compress() and compress_batch() methods

The compression store implementation in headroom/cache/compression_store.py provides hash-based retrieval for cached compressions, enabling the CCR (Compression Cache and Retrieval) system to serve repeat queries without re-computation.

Unit tests in tests/test_transforms/test_kompress_compressor.py verify importability without PyTorch dependencies, passthrough behavior when disabled, and core compression logic.

Practical Usage Examples

Compress a single long tool output:

from headroom.transforms.kompress_compressor import KompressCompressor

compressor = KompressCompressor()
result = compressor.compress(
    """This is a very long JSON payload returned by a tool call ...
    (imagine many lines of diagnostics)"""
)

print("Compressed:", result.compressed)
print(f"Saved {result.tokens_saved} tokens ({result.savings_percentage:.1f}%).")

Process multiple responses simultaneously using batch inference:

texts = ["...first long response...", "...second long response..."]
batch_results = compressor.compress_batch(texts)

for r in batch_results:
    print(r.compression_ratio, r.compressed[:100])

Retrieve previously compressed content via CCR hash:

from headroom.cache.compression_store import get_compression_store

store = get_compression_store()
original, compressed = store.retrieve("0a1b2c3d4e5f...")
print(compressed)

Summary

• Headroom's Kompress compressor uses a dual-head ModernBERT architecture to classify tokens for retention or removal, achieving 80-95% compression with minimal semantic loss.
• The system combines token-level binary classification (self.token_head) with span-importance scoring (self.span_conv) to preserve contextual coherence.
• Three-stage pipeline includes model loading (ONNX/PyTorch backends), chunked inference with 350-word segments, and CCR caching via _store_in_ccr.
• Install via pip install "headroom-ai[ml]" and configure using the HEADROOM_KOMPRESS_BACKEND environment variable.
• Core implementation is located in headroom/transforms/kompress_compressor.py with caching support in headroom/cache/compression_store.py.

Frequently Asked Questions

How does Kompress differ from standard text compression algorithms?

Unlike general-purpose compression algorithms that operate on byte sequences, Kompress performs semantic token classification using ModernBERT to identify which tokens can be removed without affecting LLM comprehension. This approach targets the specific needs of language model context windows rather than file-size reduction, typically achieving 80-95% token reduction while maintaining ≤5% meaning loss for downstream tasks.

What hardware backends does Kompress support?

Kompress supports three backend modes controlled via the HEADROOM_KOMPRESS_BACKEND environment variable: ONNX for lightweight CPU inference, PyTorch for CUDA or Apple Silicon (MPS) acceleration, and disable for passthrough mode. The system auto-detects available hardware and can utilize GPU batch processing for approximately 2× speedup when compressing three or more texts concurrently.

Where is the compressed data cached and how is it retrieved?

Compressed results meeting the CCR threshold (compression ratio < 0.8) are stored via the _store_in_ccr method in headroom/cache/compression_store.py. The system generates a hash-based cache key appended to the output. Retrieve cached content using get_compression_store().retrieve("hash_key"), which returns both the original and compressed text without re-running the ModernBERT model.

Can I use Kompress without installing PyTorch?

Yes. The tests/test_transforms/test_kompress_compressor.py verifies that Kompress remains importable and functional without PyTorch installed by using the ONNX backend or disabled mode. Install the base headroom-ai package without the [ml] extra for systems where PyTorch dependencies are undesirable, though this limits you to the disable backend or requires manual ONNX runtime setup.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}