How the Kompress‑Base ML Model Compresses Text: Token‑Level Classification with Span Boosting

The Kompress‑base model uses a dual‑head ModernBERT architecture to classify each token as keep or discard, then enhances decisions with span‑level importance scoring to produce compressed text.

The Kompress‑base model (hosted at HuggingFace ID chopratejas/kompress‑v2‑base) is the core compression engine in the chopratejas/headroom repository. Unlike traditional extractive summarizers that rely on heuristic sentence selection, this neural text compressor operates at the token level, making binary keep/discard decisions while leveraging contiguous span information to preserve semantic coherence. This article breaks down exactly how the model architecture, decision logic, and runtime pipeline work together to achieve efficient text compression.

Model Architecture: Dual‑Head ModernBERT Design

The Kompress‑base architecture is defined in HeadroomCompressorModel at lines 89‑112 of headroom/transforms/kompress_compressor.py. The implementation uses three key components:

• ModernBERT encoder – Initialized via AutoModel.from_pretrained, this produces 768‑dimensional hidden representations for each input token
• Token classification head (self.token_head) – A binary classifier that outputs logits for keep vs discard decisions per token
• Span convolution head (self.span_conv) – A 1‑D convolutional network that scores contiguous token spans, enabling the model to identify and preserve important multi‑token phrases

This dual‑head design allows the model to consider both individual token importance and contextual span information when making compression decisions.

Token‑Level Decision Logic with Borderline Handling

The compression decision pipeline lives in get_keep_mask (lines 110‑132). The logic follows a three‑stage process:

1. Hard token decisions – The token head produces logits; tokens with higher keep logit are marked for retention (token_keep)
2. Borderline identification – Tokens with keep probability between 0.3 and 0.5 enter a "borderline" region where the decision is uncertain
3. Span boost rescue – If the span head marks the same borderline region as important (span_boost), those tokens get promoted to the keep set

The final keep mask is the union of hard token decisions and borderline‑plus‑span rescues. This approach prevents the model from dropping semantically important tokens that might have marginal individual scores but high contextual relevance within key spans.

Ratio‑Based Compression via Token Scoring

When a target compression ratio is specified, the model switches to scoring mode via get_scores (lines 134‑139). The per‑token scoring formula combines both heads:

# Token probability from the binary classification head

token_prob = token_head_output

# Span boost factor (0.5 base + 0.5 * normalized span score)

span_boost = 0.5 + 0.5 * span_scores

# Final score is the product

final_score = token_prob * span_boost

The compressor then ranks tokens by this composite score and keeps the highest‑scoring tokens until the requested ratio is achieved. This scoring mechanism ensures that tokens within important spans receive proportional boost while maintaining the probabilistic foundation from the token head.

Runtime Pipeline: Chunking and Model Loading

The compress method (lines 158‑210) handles the end‑to‑end compression workflow:

Input Chunking

Long texts are split into chunks of 350 words by default to manage memory and computational constraints. Each chunk is processed independently, then reassembled into the final compressed output.

Model Loading Optimization

The _load_kompress function (lines 278‑311) implements automatic runtime selection:

• ONNX runtime – Used when available for faster inference with lower memory overhead
• PyTorch fallback – Used when ONNX is not available, with automatic CUDA detection for GPU acceleration

This dual‑path loading ensures optimal performance across different deployment environments.

Batch Processing

For high‑throughput scenarios, the compress_batch method processes multiple texts in parallel, automatically leveraging GPU acceleration when PyTorch with CUDA is available.

Post‑Processing and Compression Cache

When the resulting compression ratio falls below 0.8, the compressor optionally stores results in the Compression Cache Repository (CCR) via _store_in_ccr (lines 800‑815). This caching mechanism:

• Avoids redundant re‑compression of similar texts
• Appends retrieval hints to cached entries for faster lookup
• Integrates with the broader Headroom caching infrastructure defined in headroom/cache/compression_store.py

Practical Usage Examples

Basic Single‑Text Compression

from headroom.transforms.kompress_compressor import KompressCompressor

# Initialize with default config (uses HF_MODEL_ID)

compressor = KompressCompressor()

long_text = "The quick brown fox jumps over the lazy dog. " * 20

result = compressor.compress(long_text)
print(f"Compressed: {result.compressed}")
print(f"Ratio: {result.compression_ratio}")

Custom Model and Aggressive Compression

from headroom.transforms.kompress_compressor import KompressCompressor, KompressConfig

config = KompressConfig(
    model_id="chopratejas/kompress-finance",  # Domain‑specific variant

    chunk_words=100,                          # Smaller chunks for long documents

    score_threshold=0.45                      # More aggressive compression

)
compressor = KompressCompressor(config)

# Force 30% retention rate

result = compressor.compress(long_text, target_ratio=0.3)

Batch Processing with GPU Acceleration

texts = [
    "First long chat log with technical details...",
    "Second long tool output with execution traces...",
    "Third verbose documentation excerpt..."
]

# Automatic GPU utilization when available

batch_results = compressor.compress_batch(texts, target_ratio=0.4)

for r in batch_results:
    print(f"{r.compression_ratio:.2f}: {r.compressed[:80]}...")

Key Implementation Files

The complete Kompress‑base implementation spans these source files:

• headroom/transforms/kompress_compressor.py – Core HeadroomCompressorModel class, KompressCompressor transform, and compression logic
• headroom/transforms/base.py – Abstract Transform base class that defines the interface
• headroom/config.py – TransformResult dataclass used for structured output
• headroom/cache/compression_store.py – CCR cache implementation for storing compressed results
• scripts/export_kompress_v2_onnx.py – ONNX export script for optimized inference

Summary

• Kompress‑base uses ModernBERT with dual heads: a token classifier for binary keep/discard decisions and a span convnet for contextual importance scoring
• Decision logic combines hard thresholds with span boosting: borderline tokens (probability 0.3‑0.5) get rescued if the span head marks them important
• Ratio‑based compression uses multiplicative scoring: final token scores combine token probability with span boost factors (0.5 + 0.5 × span_score)
• Runtime optimization includes ONNX/PyTorch auto‑selection and 350‑word chunking for efficient memory usage
• Optional CCR caching stores results when compression ratio < 0.8 for retrieval‑augmented scenarios

Frequently Asked Questions

What makes Kompress‑base different from traditional extractive summarization?

Traditional extractive summarizers select entire sentences or paragraphs based on heuristic features like position or term frequency. Kompress‑base operates at the token level, allowing it to preserve critical phrases within otherwise discardable sentences. The dual‑head architecture also captures span‑level dependencies that pure sentence‑selectors miss, producing more coherent compression at arbitrary ratios.

How does the model handle very long documents?

The compress method implements automatic chunking at 350 words (configurable via chunk_words). Each chunk is processed independently through the ModernBERT encoder, then outputs are concatenated. This sliding‑window approach avoids the quadratic memory scaling of full‑document self‑attention while maintaining local coherence. For GPU batch processing, compress_batch parallelizes chunks across available CUDA devices.

Can I use Kompress‑base without HuggingFace dependencies?

Yes. The runtime automatically selects ONNX inference when available, eliminating the PyTorch/HuggingFace dependency chain for production deployment. Run scripts/export_kompress_v2_onnx.py to convert the HuggingFace checkpoint to ONNX format. The _load_kompress function (lines 278‑311) handles ONNX Runtime initialization with graceful fallback to PyTorch when needed.

What compression ratios are practical with this model?

The model supports arbitrary target ratios via the target_ratio parameter (0.0 to 1.0). In practice, ratios between 0.3 and 0.6 maintain readable output for most domains. Below 0.3, coherence degrades as the span head cannot compensate for excessive token dropping. The default score_threshold of 0.5 provides balanced compression; lowering to 0.3‑0.4 increases aggressiveness at the cost of occasional semantic drift.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}