How RTK and Caveman Compression Save Tokens in OmniRoute

TLDR: OmniRoute reduces LLM token consumption by approximately 78%–95% by chaining the RTK (Run-Time-Kit) engine for command-aware compression with the Caveman engine for prose-aware compression, applying cascaded filters, deduplication, and intensity-based truncation to tool outputs before they reach the model.

OmniRoute is an open-source routing layer that dramatically reduces LLM costs by intercepting and compressing noisy machine-generated outputs before they enter the context window. By combining two specialized compression engines—RTK for structured command output and Caveman for narrative text—the system achieves multiplicative token savings while preserving critical error information and semantic intent. This article explains how RTK and Caveman compression save tokens based on the actual implementation in the diegosouzapw/OmniRoute repository.

Understanding the RTK Compression Engine

The RTK (Run-Time-Kit) engine provides command-aware compression that targets noisy machine-generated output from tools like git, npm test, Docker, and build runners. According to the RTK benchmark data cited in docs/compression/RTK_COMPRESSION.md, this engine alone achieves 60%–90% token reduction, averaging approximately 80% savings on typical CI/CD and development output.

Command Detection and Filter Loading

In open-sse/services/compression/engines/rtk/commandDetector.ts, the system classifies incoming tool output to determine the appropriate compression strategy. The engine then loads filters in a specific hierarchy:

1. Project-level filters from .rtk/filters.*
2. Global filters from DATA_DIR/rtk/filters.*
3. Built-in filters from open-sse/services/compression/engines/rtk/filters/

Each filter follows the JSON schema defined in open-sse/services/compression/engines/rtk/filterSchema.ts, specifying rules for stripping ANSI codes, pattern matching, inclusion/exclusion logic, and truncation.

Filter DSL and Deduplication

The RTK engine applies a cascade of filters that perform text transformations without sending raw output to the LLM (unless rawOutputRetention is enabled for debugging). Key mechanisms include:

• Rule-based deduplication: Individual filters can specify rules.deduplicate to collapse repeated lines within their scope.
• Engine-wide deduplication: A final pass in open-sse/services/compression/engines/rtk/index.ts collapses runs of identical consecutive lines using deduplicateThreshold (default: 3 lines).
• Similarity grouping: When rtkConfig.enableGrouping is true, the groupSimilarLines function collapses near-identical consecutive lines into single representative entries.

Intensity Levels and Truncation

RTK supports three intensity levels that control truncation aggressiveness:

• minimal: Preserves most content with light truncation
• standard (default): Truncates sections to approximately 24 lines per section
• aggressive: Truncates sections to approximately 16 lines per section, maximizing token savings for high-volume outputs

Understanding the Caveman Compression Engine

Caveman serves as the second-stage prose-aware compressor that processes RTK-completed output to remove narrative noise. Located in open-sse/services/compression/caveman.ts, this engine applies heuristics specifically tuned for LLM-friendly summarization, achieving an additional ~46% reduction on the already-compressed RTK output.

Prose-Aware Heuristics

The compression rules defined in open-sse/services/compression/cavemanRules.ts perform several text optimization strategies:

• Hedging removal: Strips filler phrases (e.g., "as far as I can tell") that add token overhead without semantic value
• Structural pruning: Collapses repetitive stack traces and condenses long log sequences
• Error preservation: Maintains critical error messages, stack traces, and test failure details essential for LLM reasoning

Secondary Deduplication

After applying per-rule filters, Caveman runs a final deduplication pass using the same threshold (3 identical consecutive lines) as the RTK engine. This ensures that any repetitive patterns surviving the first stage are eliminated before the content enters the LLM context window.

Calculating Stacked Token Savings

When the two engines are chained (rtk → caveman), the combined effect is multiplicative rather than additive:

Stacked saving = 1 – (1 – RTK_save) × (1 – Caveman_save)
               = 1 – (1 – 0.80) × (1 – 0.46)
               ≈ 89% overall token reduction

Based on the RTK variability range (60%–90%), the overall token savings fall between 78% and 95%.

Engine	Compression Strategy	Typical Savings
RTK	Command-aware filtering and truncation	~80% (60%–90% range)
Caveman	Prose-aware summarization and hedging removal	~46% of RTK output
Combined	Stacked pipeline	~89% (78%–95% range)

Configuring Compression Pipelines

You can interact with these engines via the REST API or programmatic configuration.

Preview Stacked Compression

To test the combined pipeline before deployment:

POST /api/compression/preview
{
  "mode": "stacked",
  "messages": [
    {
      "role": "tool",
      "content": "FAIL tests/example.test.ts\nAssertionError: expected true\nTest Files 1 failed"
    }
  ],
  "config": {
    "rtkConfig": { "intensity": "standard" },
    "cavemanConfig": { "intensity": "full" }
  }
}

The response includes the compressed payload and a CompressionStats object detailing individual savings from each engine.

Adjusting Intensity for Maximum Savings

For high-volume environments, configure aggressive compression:

import { updateEngineConfig } from "@omniroute/open-sse/services/compression/engines/registry";

updateEngineConfig("rtk", { intensity: "aggressive" });
updateEngineConfig("caveman", { intensity: "full" });

Setting RTK to aggressive reduces section limits to 16 lines, while full Caveman intensity applies all available heuristics, pushing total reduction toward the ~95% upper bound.

Retaining Raw Output for Debugging

To preserve uncompressed copies for audit purposes:

PUT /api/context/rtk/config
{
  "rawOutputRetention": "always",
  "maxCharsPerResult": 12000
}

Raw outputs are stored in DATA_DIR/rtk/raw-output/ and accessible via:

GET /api/context/rtk/raw-output/<id>

Summary

• RTK compression (in open-sse/services/compression/engines/rtk/index.ts) provides command-aware filtering that reduces tokens by 60%–90% through pattern matching, deduplication, and configurable intensity truncation.
• Caveman compression (in open-sse/services/compression/caveman.ts) applies prose-aware heuristics that further reduce the RTK-compressed payload by approximately 46%.
• Stacked configuration achieves 78%–95% total token reduction by multiplicatively combining both engines.
• Configuration persistence is handled in src/lib/db/compression.ts using an SQLite key_value table.
• Quality assurance is maintained through the golden set tests in tests/golden-set/compression-caveman-v2.test.ts.

Frequently Asked Questions

What is the typical token reduction when using both RTK and Caveman compression?

When chained together, RTK and Caveman compression save approximately 89% of tokens on average, with a range between 78% and 95% depending on the input noise level and selected intensity settings. RTK handles the initial heavy lifting on structured command output, while Caveman optimizes the remaining prose.

How does the RTK engine determine which filters to apply?

The commandDetector.ts module classifies incoming tool output (e.g., git diff, npm test) and loads appropriate filters in order: project-level (.rtk/filters.*), global (DATA_DIR/rtk/filters.*), and finally built-in defaults. Each filter is validated against the schema in filterSchema.ts before application.

What is the difference between RTK and Caveman compression?

RTK is command-aware, designed for structured machine output like logs and diffs, using pattern matching and line-based deduplication. Caveman is prose-aware, designed for narrative text, using heuristics to remove hedging language and collapse repetitive phrases while preserving critical error context.

Can I view the original uncompressed output after RTK compression?

Yes. Setting rawOutputRetention to "always" in the RTK configuration (stored via src/lib/db/compression.ts) preserves redacted originals in DATA_DIR/rtk/raw-output/. You can retrieve these using the GET /api/context/rtk/raw-output/<id> endpoint for debugging or verification against the compressed results.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}