# How Kompress-base HuggingFace Model Compression Works in Headroom: A Technical Deep Dive

> Discover how Headroom's Kompress-base HuggingFace model compression works. This technical deep dive explains its multi-stage pipeline for intelligent token pruning and efficient long text processing.

- Repository: [Tejas Chopra/headroom](https://github.com/chopratejas/headroom)
- Tags: deep-dive
- Published: 2026-06-14

---

**Headroom's Kompress transform automatically downloads a ModernBERT-based model from HuggingFace (`chopratejas/kompress-v2-base`) and applies intelligent token pruning to long tool outputs and assistant messages through a multi-stage pipeline involving chunking, dual-head neural inference, and cache-controlled retrieval.**

The Kompress-base HuggingFace model compression system serves as the core text reduction engine within the Headroom framework. Located in the `chopratejas/headroom` repository, this implementation processes lengthy generated content by identifying semantically critical tokens while discarding redundant information. The architecture supports multiple execution backends and includes privacy-preserving caching mechanisms to optimize for both latency and computational efficiency.

## Model Architecture and Backend Selection

The compression system centers on a **dual-head ModernBERT architecture** that provides flexible inference strategies based on the execution environment.

### The ModernBERT Base and Dual-Head Design

In [`headroom/transforms/kompress_compressor.py`](https://github.com/chopratejas/headroom/blob/main/headroom/transforms/kompress_compressor.py), the `HeadroomCompressorModel` class (defined in `_get_model_class()` at lines 83-115) extends the standard ModernBERT architecture with two specialized heads. The first head generates binary **keep-masks** through `get_keep_mask()` (lines 118-132), determining whether individual tokens should be retained. The second head employs a 1-D CNN to produce **span-importance scores** via `get_scores()` (lines 134-141), ranking token significance for ratio-based compression.

### Backend Abstraction: ONNX vs. PyTorch

The system supports multiple execution backends through the `KompressBackend` literal type, allowing deployment flexibility. The `_selected_backend()` function (lines 100-117) determines whether to load the ONNX runtime, PyTorch weights, or CoreML variants based on the `HEADROOM_KOMPRESS_BACKEND` environment variable.

The `_load_kompress()` orchestrator (lines 122-165) manages lazy downloading of model artifacts from HuggingFace. For ONNX execution, `_load_kompress_onnx()` (lines 372-442) initializes an `_OnnxModel` wrapper (lines 148-166) that provides a unified interface matching the PyTorch implementation. PyTorch execution flows through `_load_kompress_pytorch()` (lines 506-558), which handles device placement and GPU optimization.

## The 9-Step Compression Pipeline

The `compress()` method implements a sophisticated multi-stage pipeline that processes text while respecting the model's 512-token limit.

### 1. Model Selection and Lazy Loading

The pipeline begins with backend selection and lazy model initialization. When `allow_download=False` is specified, the system operates in cache-only mode, avoiding network I/O during startup. The `_load_modernbert_tokenizer()` function (lines 494-503) concurrently loads the `answerdotai/ModernBERT-base` tokenizer with the same caching semantics.

### 2. Text Chunking

Input text is segmented into **chunks of 350 words** (configurable via `self.config.chunk_words`) to ensure each inference request remains under the 512-token limit. The chunking logic (lines 88-90) calculates `chunk_start` and `chunk_words` indices to partition long documents without breaking semantic continuity.

### 3. Token Encoding

For each chunk, the tokenizer returns `input_ids`, `attention_mask`, and a critical `word_ids` mapping that links token positions back to original word indices. ONNX backends receive NumPy tensors, while PyTorch uses `torch.Tensor` objects.

### 4. Dual-Mode Inference

The model executes either `get_keep_mask()` for binary classification or `get_scores()` for importance ranking, depending on whether a `target_ratio` compression parameter is supplied. This selection determines whether the system uses threshold-based or ratio-based token retention.

### 5. Token-to-Word Mapping

Using the `word_ids` mapping, token-level predictions collapse to word-level decisions. For threshold-based compression, words with scores exceeding **0.5** (configurable via `score_threshold`) are retained. For ratio-based compression, the top-k words by score are selected to meet the target ratio.

### 6. Text Reassembly

Kept word indices are sorted and joined into the compressed output string. The system calculates the compression ratio, and if the reduction exceeds 20% (ratio < 0.8), the result is stored in the **CCR (Cache-Controlled Retrieval)** system via `_store_in_ccr()` (lines 1200-1210).

### 7. Privacy-Preserving Caching

The CCR system uses `_kompress_content_signature()` (lines 202-237) to generate privacy-preserving signatures for cache keys, ensuring sensitive content remains secure while enabling retrieval of previously computed compressions.

### 8. Telemetry and Performance Monitoring

The pipeline emits slow-path warnings when inference exceeds 1000ms and optionally records compression metrics through `record_compression` telemetry hooks (lines 100-110).

### 9. Batch Optimization

For GPU-enabled PyTorch backends, `compress_batch()` (lines 95-110) aggregates multiple chunks into single forward passes (batch size ≈ 32), providing approximately 2× speedup for multi-document processing. The `_should_use_sequential_fallback()` helper (lines 120-125) determines when to batch versus process sequentially.

## Configuration and Customization

The `KompressConfig` class (lines 286-306) exposes granular control over compression behavior:

- **model_id**: Specify alternative HuggingFace models (e.g., `chopratejas/kompress-finance`)
- **chunk_words**: Adjust chunk size (default 350)
- **score_threshold**: Modify retention aggressiveness (default 0.5)
- **enable_ccr**: Toggle Cache-Controlled Retrieval
- **device**: Target specific GPU/CPU placement

## Implementation Examples

### Basic Single-String Compression

```python
from headroom.transforms.kompress_compressor import KompressCompressor

# Initialize with default kompress-v2-base model

compressor = KompressCompressor()

# Compress lengthy tool output

long_text = "Detailed assistant response exceeding token limits..."
result = compressor.compress(long_text)

print(f"Compressed: {result.compressed}")
print(f"Ratio: {result.compression_ratio}")

```

### Backend Selection and GPU Acceleration

```python
import os

# Configure execution backend

os.environ["HEADROOM_KOMPRESS_BACKEND"] = "pytorch"  # or "onnx", "onnx_coreml"

os.environ["CUDA_VISIBLE_DEVICES"] = "0"

compressor = KompressCompressor()
result = compressor.compress("Very long text...")

```

### Batch Processing for High Throughput

```python
texts = [
    "First assistant message...",
    "Second tool output...",
    "Third document..."
]

compressor = KompressCompressor()
results = compressor.compress_batch(texts)

for r in results:
    print(f"Achieved ratio: {r.compression_ratio}")

```

### Custom Model Configuration

```python
from headroom.transforms.kompress_compressor import KompressConfig, KompressCompressor

cfg = KompressConfig(
    model_id="chopratejas/kompress-finance",
    chunk_words=200,
    score_threshold=0.45,
    enable_ccr=False
)

compressor = KompressCompressor(cfg)

```

## Summary

- **Headroom's Kompress transform** utilizes a ModernBERT-based dual-head architecture from HuggingFace (`chopratejas/kompress-v2-base`) to compress long tool outputs and assistant messages.
- The system supports **multiple backends** (ONNX, PyTorch, CoreML) through environment variable configuration, enabling deployment flexibility across CPU and GPU environments.
- **Chunk-based processing** (default 350 words) ensures compliance with the 512-token limit while maintaining semantic coherence through token-to-word mapping.
- **Dual inference modes** provide either binary keep/discard decisions or ratio-based importance scoring, configurable via `target_ratio` and `score_threshold` parameters.
- **Privacy-preserving CCR caching** stores compression results when efficiency exceeds 20%, using content signatures to avoid recomputing similar texts.
- **Batch optimization** in PyTorch GPU mode delivers approximately 2× throughput improvements for multi-document workloads.

## Frequently Asked Questions

### How does Headroom choose between ONNX and PyTorch backends?

The `_selected_backend()` function in [`headroom/transforms/kompress_compressor.py`](https://github.com/chopratejas/headroom/blob/main/headroom/transforms/kompress_compressor.py) reads the `HEADROOM_KOMPRESS_BACKEND` environment variable to determine execution mode. Valid options include `"onnx"`, `"pytorch"`, `"onnx_coreml"`, and `"auto"`. ONNX provides lighter CPU inference, while PyTorch enables GPU acceleration and batch processing capabilities.

### What is the maximum text length the Kompress model can handle?

The underlying ModernBERT model has a 512-token limit per inference pass. To handle longer texts, Headroom implements **chunking** that splits input into 350-word segments (configurable via `chunk_words` in `KompressConfig`). This ensures each chunk fits within model constraints while preserving enough context for accurate importance scoring.

### How does the compression ratio target work?

When `target_ratio` is specified, the system uses `get_scores()` to rank all tokens by importance, then retains the top-k words to achieve the desired compression percentage. Without a target ratio, the binary `get_keep_mask()` applies a threshold (default 0.5) to determine token retention. The `score_threshold` parameter in `KompressConfig` adjusts this aggressiveness.

### What is CCR caching and when does it activate?

**CCR (Cache-Controlled Retrieval)** stores original-compressed text pairs when the compression ratio drops below 0.8 (indicating >20% reduction). The `_store_in_ccr()` function uses `_kompress_content_signature()` to generate privacy-preserving cache keys. This prevents recomputation of frequently occurring long outputs while ensuring sensitive data remains secure through signature-based indexing rather than raw text storage.