How RTK + Caveman Stacked Compression Saves Tokens in Tool-Heavy Sessions

OmniRoute's stacked compression pipeline chains RTK (fast rule-based filtering) with Caveman (model-aware semantic compression) to reduce token counts by over 90% in tool-heavy sessions while preserving critical diagnostic information.

The diegosouzapw/OmniRoute repository implements a two-stage compression system specifically designed for LLM interactions that generate large, repetitive tool outputs. When compression.mode is set to "stacked", the pipeline automatically routes messages through RTK followed by Caveman, stripping both syntactic and semantic redundancy before the payload reaches the model.

---

What Is Stacked Compression in OmniRoute?

Stacked compression is a pipeline architecture that processes messages through multiple specialized engines in sequence. In open-sse/services/compression/types.ts, the CompressionConfig interface defines mode: "stacked" as the trigger for this multi-stage behavior.

Unlike single-engine compression, stacking allows each engine to target a distinct class of redundancy. RTK handles deterministic, pattern-based reduction. Caveman then operates on the pre-cleaned output to apply model-aware scoring and relevance-based pruning.

This division of labor matters because tool-heavy sessions—build logs, file reads, CLI output—produce payloads with two distinct problems: repetitive formatting noise and low-information semantic content. A single-pass compressor must either miss one or over-aggressively damage the other.

---

Stage 1: RTK Rapid Tokenizer Engine

RTK (open-sse/services/compression/engines/rtkEngine.ts) is the fast, deterministic first pass. It applies a catalog of filters that require no model inference and complete in milliseconds.

Key RTK Operations

• maxLinesPerResult and maxCharsPerResult: Hard truncates oversized tool_result blocks at configurable boundaries
• deduplicate: true: Removes identical consecutive lines, eliminating repetitive log output
• Comment stripping: Removes code comments from tool-generated snippets
• Force-preserve rules: Protects safety-critical tokens including URLs, API keys, and specific regex patterns

The engine's output is a trimmed payload that retains semantic content while discarding structural noise. Tests in tests/unit/compression/rtk-smart-truncate.test.ts verify this behavior across RTK smart-truncate, RTK deduplication, and RTK filter catalog scenarios.

---

Stage 2: Caveman Model-Aware Compression

Caveman receives RTK's output through open-sse/services/compression/engines/cavemanAdapter.ts. This second stage performs deeper, context-sensitive reduction that requires understanding token information value.

Caveman Compression Strategies

• Relevance scoring: Weighs each token against session context and model-specific importance patterns
• Session-dedup: Eliminates semantic duplicates even when phrasing differs
• Headroom trimming: Aggressive reduction when approaching token budget limits
• Hard-budget guard: Prevents over-truncation that would break output coherence
• Fidelity guard: Validates that final output satisfies target token constraints

The intensity configuration (set to "high" in demanding scenarios) directly influences scoring thresholds. High-value tokens—URLs, numeric identifiers, preserved docstrings—receive protected status regardless of intensity settings.

---

Combined Token Savings: The 90% Reduction Mechanism

When RTK and Caveman execute in sequence, each removes a non-overlapping category of redundancy:

Stage	Redundancy Type	Typical Reduction	Mechanism
RTK	Syntactic repetition (duplicate lines, large unstructured blobs)	40-60%	Deterministic filtering
Caveman	Semantic redundancy (low-scoring tokens, contextual irrelevance)	30-50% of remaining	Model-aware scoring

The multiplicative effect frequently exceeds 90% total token savings on deliberately noisy tool_result blocks. This is demonstrated in tests/unit/compression/stacked-compression-tool-result-savings.test.ts, which validates the pipeline against realistic tool-heavy payloads that would otherwise overwhelm standard LLM context windows.

---

Why Tool-Heavy Sessions Require Stacked Compression

Tool-heavy interactions generate three specific problems that stacked compression solves:

1. Volume: Build logs and file listings quickly exceed 100K+ tokens
2. Repetition: CLI output contains massive structural duplication
3. Noise density: High token count with low information-per-token ratio

By applying RTK first, OmniRoute stays within provider token budgets without truncation errors. Caveman then ensures the LLM receives the maximum-information subset of available output. The pipeline remains deterministic for safety-critical tokens through operator-defined preservation rules, satisfying security compliance requirements.

---

Enabling and Configuring Stacked Compression

Basic Activation

// Enable stacked compression via request payload
await fetch("/api/v1/chat/completions", {
  method: "POST",
  headers: { "Content-Type": "application/json" },
  body: JSON.stringify({
    model: "gpt-4o-mini",
    messages: [
      { role: "user", content: "Run `git status` and show the result." },
    ],
    compression: { mode: "stacked" },
  }),
});

Fine-Grained Configuration

// Persistent configuration for both engines
const DEFAULT_RTK_CONFIG = {
  enabled: true,
  maxLinesPerResult: 120,
  deduplicate: true,
};

const DEFAULT_CAVEMAN_CONFIG = {
  enabled: true,
  intensity: "high",
  preserveDocstrings: true,
};

await fetch("/api/v1/compression/config", {
  method: "POST",
  headers: { "Content-Type": "application/json" },
  body: JSON.stringify({
    rtkConfig: DEFAULT_RTK_CONFIG,
    cavemanConfig: DEFAULT_CAVEMAN_CONFIG,
  }),
});

The configuration service persists settings across requests, while per-request compression.mode overrides allow session-specific pipeline selection.

---

Core Implementation Files

Component	Path	Function
Mode resolution	open-sse/services/compression/resolveCompressionPlan.ts	Selects between off, rtk, codex-responses, stacked
Pipeline orchestration	open-sse/services/compression/strategySelector.ts	Builds ordered engine list (rtk → caveman)
RTK engine	open-sse/services/compression/engines/rtkEngine.ts	Line-filtering, deduplication, truncation
Caveman adapter	open-sse/services/compression/engines/cavemanAdapter.ts	Stacked-stage wrapper for Caveman
Staged execution	open-sse/services/compression/stackedStepCore.ts	Progress tracking, bail-out guards, telemetry
Savings validation	tests/unit/compression/stacked-compression-tool-result-savings.test.ts	>90% reduction verification
RTK correctness	tests/unit/compression/rtk-smart-truncate.test.ts	Filter and dedup validation
Scoring logic	tests/unit/compression/heatmap-build.test.ts	Token score generation testing

---

Summary

• RTK + Caveman stacked compression in OmniRoute achieves 90%+ token reduction by chaining deterministic filtering with model-aware semantic pruning
• RTK (rtkEngine.ts) handles syntactic noise: line limits, deduplication, comment stripping, with force-preserve rules for safety tokens
• Caveman (cavemanAdapter.ts) scores remaining tokens by relevance, applying intensity-configurable thresholds and budget guards
• The compression.mode: "stacked" trigger activates this pipeline through strategySelector.ts and stackedStepCore.ts
• Tool-heavy sessions benefit most: build logs, CLI output, and file reads stay within context windows without losing diagnostic value
• Full test coverage in stacked-compression-tool-result-savings.test.ts validates real-world savings

---

Frequently Asked Questions

What triggers the stacked compression pipeline?

The pipeline activates when a request includes compression: { mode: "stacked" }. resolveCompressionPlan.ts parses this configuration and strategySelector.ts constructs the ordered engine list. No server-side configuration changes are required—clients opt-in per-request or persist preferences via the /api/v1/compression/config endpoint.

How does RTK protect important tokens while still saving space?

RTK implements force-preserve rules—regex patterns that mark URLs, API keys, and operator-defined critical strings as non-truncatable. These rules run before any filtering or deduplication. The engine also respects maxLinesPerResult boundaries that truncate aggressively but predictably, avoiding mid-token splits that could corrupt preserved content.

When should I use stacked versus single-engine compression?

Use stacked for tool-heavy sessions with large, repetitive outputs (builds, logs, bulk file operations) where both volume and noise density are high. Use RTK-only for simple truncation needs where speed matters more than maximum compression. Use Caveman-only when inputs are already clean but require semantic relevance scoring. The codex-responses mode provides an alternative pipeline optimized for OpenAI-style streaming responses.

Can I adjust the 90% savings target?

Yes. Tune maxLinesPerResult and maxCharsPerResult in RTK config to control first-pass aggression. Adjust Caveman's intensity ("low", "medium", "high") to shift scoring thresholds. The hard-budget and fidelity guards in stackedStepCore.ts ensure these adjustments don't produce incoherent output—they trigger early termination or fallback to less aggressive modes rather than violate safety constraints.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}