OmniRoute Token Compression Pipeline (RTK + Caveman): Achieving Up to 95% Token Savings

OmniRoute reduces LLM token usage by up to 95% through a stacked compression pipeline that first processes command output with RTK (Rust Token Killer) for 60-90% savings, then applies Caveman's language-aware compression for an additional ~46% reduction on the remaining text.

The OmniRoute token compression pipeline combines two specialized engines to drastically reduce the token count sent to large language models during coding agent sessions. By stacking these compressors sequentially as implemented in diegosouzapw/OmniRoute, the system achieves compounding savings that lower API costs and prevent context window overflow. This architecture specifically targets the noisy tool output and verbose prose that dominate software engineering workflows.

Understanding the RTK + Caveman Architecture

What is RTK (Rust Token Killer)?

RTK is a command-aware compressor that targets noisy tool and build output including git logs, test results, shell transcripts, and Docker logs. Located in open-sse/services/compression/engines/rtk/index.ts, the engine strips ANSI color codes, applies filter-specific regex replacements, deduplicates repeated lines, and truncates verbose sections.

Upstream RTK reports savings between 60-90% on raw command output, with OmniRoute using an average figure of approximately 80% for internal calculations. The engine dynamically loads filter catalogs through open-sse/services/compression/engines/rtk/filterLoader.ts, which aggregates built-in, global, and project-level filter definitions to handle domain-specific output formats.

What is Caveman?

Caveman is a language-aware prose compressor that applies rule packs for English, French, German, Portuguese, Chinese, Japanese, and other languages. Implemented in open-sse/services/compression/caveman.ts, this engine applies linguistic compression rules defined in open-sse/services/compression/cavemanRules.ts to reduce token count while preserving semantic meaning and technical terminology.

When RTK-compressed text feeds into Caveman, the engine reduces the remaining token count by roughly 46%. Caveman supports intensity levels ranging from lite to ultra, allowing fine-tuned trade-offs between compression aggressiveness and information density.

How Stacked Compression Calculates Savings

The default stacked mode processes text through RTK first, then pipes the output into Caveman. This sequential processing compounds savings multiplicatively rather than additively. According to docs/compression/RTK_COMPRESSION.md, the calculation follows this formula:

Stacked savings = 1 – (1 – RTK) × (1 – Caveman_input)
                ≈ 1 – (0.20) × (0.54) 
                = 0.892  →  89.2% saved

Accounting for upstream RTK variance (60-90%), the realistic savings range spans 78% to 95% token reduction. Production coding-agent sessions typically observe 80-89% overall compression, dramatically lowering model-usage costs while preserving critical error information and code blocks.

Configuring the Stacked Pipeline

Global Configuration Structure

The pipeline configuration resides in the global compression config via the stackedPipeline entry, with type definitions in open-sse/services/compression/types.ts. This array specifies execution order and intensity levels for each engine:

{
  "defaultMode": "stacked",
  "stackedPipeline": [
    { "engine": "rtk", "intensity": "standard" },
    { "engine": "caveman", "intensity": "standard" }
  ]
}

Intensity levels vary by engine. RTK supports minimal, standard, or aggressive, while Caveman offers lite, standard, aggressive, and ultra.

HTTP API Examples

Preview stacked compression before applying it live:

POST /api/compression/preview
{
  "mode": "stacked",
  "messages": [
    { "role": "tool", "content": "FAIL tests/example.test.ts\nAssertionError: expected true\nTest Files 1 failed" }
  ],
  "config": {
    "rtkConfig": { "intensity": "standard" },
    "cavemanConfig": { "intensity": "standard" }
  }
}

Update the global configuration to enable the default stacked pipeline:

PUT /api/context/rtk/config
{
  "defaultMode": "stacked",
  "stackedPipeline": [
    { "engine": "rtk", "intensity": "standard" },
    { "engine": "caveman", "intensity": "standard" }
  ]
}

CLI and Programmatic Usage

Switch to stacked mode via the OmniRoute CLI (v3.8.50):

omniroute compression set --mode stacked
omniroute compression preview --mode stacked --file prompt.txt

Configure programmatically in Node.js or TypeScript:

import { updateEngineConfig } from "@omniroute/open-sse/services/compression/engines/registry";

updateEngineConfig("rtk", { intensity: "standard" });
updateEngineConfig("caveman", { intensity: "standard" });

Key Implementation Files

The compression system spans several critical source files in the repository:

• open-sse/services/compression/engines/rtk/index.ts: Core RTK engine implementation handling the filter pipeline, deduplication logic, and intensity-based processing.
• open-sse/services/compression/engines/rtk/filterLoader.ts: Dynamic loading mechanism for built-in, global, and project-specific filter catalogs.
• open-sse/services/compression/caveman.ts: Caveman engine wrapper that orchestrates language rule application and optional text grouping.
• open-sse/services/compression/cavemanRules.ts: Rule metadata and language pack definitions supporting multilingual compression.
• open-sse/services/compression/types.ts: TypeScript interfaces for RtkConfig, CavemanConfig, and the combined CompressionConfig.

Summary

• RTK (Rust Token Killer) targets structured command output (logs, tests, builds) with 60-90% token savings by stripping ANSI codes and deduplicating lines.
• Caveman applies language-aware compression to prose with an additional ~46% reduction on the remaining input.
• Stacked mode compounds these savings to achieve 78-95% total token reduction, with 80-89% being typical in production coding sessions.
• Configuration occurs through the stackedPipeline array in the global compression config, allowing independent intensity controls for each engine.
• The implementation provides type-safe interfaces in open-sse/services/compression/types.ts and supports HTTP, CLI, and programmatic configuration methods.

Frequently Asked Questions

How does the OmniRoute token compression pipeline maintain data integrity with high compression rates?

The pipeline preserves critical information through domain-specific awareness. RTK uses command-specific filters in filterLoader.ts that identify and protect error messages, stack traces, and code blocks while removing ANSI codes and duplicate lines. Caveman applies linguistic rules that remove filler words without altering technical terminology or code syntax.

What is the difference between RTK and Caveman intensity levels?

RTK intensity (minimal, standard, aggressive) controls the aggressiveness of regex replacements and truncation thresholds for tool output. Caveman intensity (lite, standard, aggressive, ultra) determines how strictly linguistic rules compress prose, with ultra removing more articles and conjunctions while lite preserves natural language flow.

Can I use RTK or Caveman independently instead of the stacked pipeline?

Yes, though the stacked configuration provides optimal results. The defaultMode setting accepts individual engine names, but using RTK alone limits savings to 60-90%, while Caveman alone lacks the structured data cleaning necessary for command output. The stacked approach (rtk -> caveman) ensures clean data flows into the prose compressor for maximum efficiency.

How do I verify token savings before applying compression to live requests?

Use the preview endpoint POST /api/compression/preview or the CLI command omniroute compression preview --mode stacked --file prompt.txt. These interfaces process sample content through the configured pipeline and return before/after token counts without modifying your active session configuration.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}