How Prompt Compression Works in OmniRoute: RTK and Caveman Codecs Explained

Prompt compression in OmniRoute reduces token costs by pre-processing API requests through a stacked pipeline where the RTK engine applies semantic, command-aware filters first, followed by the Caveman engine's aggressive rule-based heuristics.

OmniRoute implements a modular compression layer that intercepts outgoing LLM requests to minimize token usage before transmission to providers. The system supports multiple compression engines that can be stacked in sequence, with RTK (Rule-Based Tool-output Killer) and Caveman serving as the two primary codecs for shrinking tool outputs and code blocks. Both engines operate on a normalized message format and report detailed statistics through the createCompressionStats() utility.

The Compression Pipeline Architecture

The compression workflow follows a strict pipeline defined in open-sse/services/compression/strategySelector.ts. This architecture ensures that transformations occur in a predictable order with full observability.

Message Adaptation

The pipeline begins with adaptBodyForCompression() located in open-sse/services/compression/bodyAdapter.ts. This function normalizes the OpenAI-compatible request shape into a uniform Message[] array, allowing downstream engines to operate on a consistent data structure regardless of the specific provider format.

Engine Selection and Stacking

The strategySelector.ts module inspects the user's compression settings—specifically engines.rtk.enabled and engines.caveman.enabled—to construct an execution pipeline. When both engines are active, the system stacks them according to their declared stackPriority values:

• RTK: stackPriority = 10
• Caveman: stackPriority = 20

Because the selector arranges engines in ascending priority order, the execution flow becomes RTK → Caveman. Each engine's apply() method receives the request body and returns a modified body along with per-engine statistics, which createCompressionStats() in open-sse/services/compression/stats.ts aggregates into a final report.

Execution Flow

Raw request 
  → adaptBodyForCompression() 
  → [RTK Engine] 
  → [Caveman Engine] 
  → restoreBody() 
  → API response with compression stats

RTK Engine: Semantic Tool-Output Compression

The RTK engine, implemented in open-sse/services/compression/engines/rtk/index.ts, targets tool_results and code_blocks using command-aware, declarative filters. It emphasizes semantic understanding over brute-force truncation.

Configuration and Eligibility

The engine initializes via mergeRtkConfig() (lines 77-104), which combines default settings with user-provided configurations for fields like intensity, maxLinesPerResult, and filter lists. The shouldCompressMessage() function (lines 27-43) determines eligibility based on applyTo… flags, ensuring the engine only processes tool results, code blocks, or assistant messages as configured.

Command Detection and Filter Matching

RTK identifies content type through detectCommandType() in commandDetector.ts, classifying inputs as docker-logs, git-diff, generic-output, or other command types. The engine then invokes matchRtkFilter() from filterLoader.ts to select a declarative filter whose id matches the detected command type and intensity level. Disabled filters are automatically skipped.

Compression Techniques

The RTK engine applies several transformation techniques in sequence:

• applyLineFilter(): Drops or trims lines according to the filter's budget (maxLines) and priority patterns, preserving high-value content like error lines.
• Semantic Renderers: When enableRenderers is true, applyRenderer() executes specialized renderers (e.g., terraformPlan, gitDiff) that rewrite verbose outputs into concise summaries.
• processRtkText(): Removes code-fence sections and optionally strips comments via stripCodeComments while preserving doc-strings (lines 76-92).
• deduplicateRepeatedLines(): Collapses repeated lines to eliminate redundancy.
• groupSimilarLines(): Merges lines with high similarity (opt-in via configuration).
• smartTruncate(): Enforces line/character budgets scaled by the intensity setting (minimal, standard, aggressive), protecting stack traces and error patterns.

Raw Output Retention

If compression yields a smaller payload, maybePersistRtkRawOutput() (lines 44-53) stores the original text for debugging, creating a pointer recorded in the statistics under techniquesUsed as rtk-raw-output-retention.

Caveman Engine: Aggressive Rule-Based Compression

Caveman serves as a fallback and aggressive secondary compressor implemented in open-sse/services/compression/engines/cavemanAdapter.ts. It executes after RTK with stackPriority = 20, operating on hard-coded heuristics rather than declarative filters.

Rule-Driven Architecture

The engine maintains a static array of rules including removeNoiseLines, collapseLargeBlocks, and stripStackTraces. Each rule specifies regex patterns, actions, and priorities. The applyCavemanCompression() function iterates through these rules, applying the first match to each line or block. Rules may trim extremely long lines, drop boilerplate sections (e.g., "Generated by..."), or preserve identifiers matching a protected whitelist.

Intensity and Safety Controls

Caveman supports three intensity levels—lite, full, and ultra—that scale the maximum allowed lines via effectiveMaxLines(). When autoClarity is enabled (default: true), the engine applies extra safety heuristics in full intensity mode to maintain essential readability. Unlike RTK, Caveman disables raw-output retention by default to minimize latency, though it can be enabled via configuration.

Reported Techniques

The engine reports specific techniques in the engineBreakdown statistics:

• caveman-noise-drop: Removes lines matching common noise patterns.
• caveman-line-trim: Cuts overly long lines to safe lengths.
• caveman-preserve-identifiers: Retains symbols from the protected list.

Engine Stacking and Interaction

When both engines are enabled, the pipeline processes content sequentially:

1. RTK First: Applies semantic renderers and command-specific filters, reducing the payload while preserving structure.
2. Caveman Second: Receives the already-compressed output and applies aggressive heuristics to remove remaining noise.

This ordering ensures that RTK's semantic understanding (e.g., recognizing a git diff and formatting it appropriately) occurs before Caveman's pattern-based truncation. If only one engine is enabled, the pipeline bypasses the other entirely. If both are disabled, requests transmit without compression.

Configuration and Usage Examples

Enable Both Engines with Aggressive RTK

{
  "engines": {
    "rtk": {
      "enabled": true,
      "intensity": "aggressive",
      "enableRenderers": true
    },
    "caveman": {
      "enabled": true,
      "intensity": "full"
    }
  }
}

This configuration runs semantic renderers and aggressive line filtering in RTK, followed by Caveman's noise removal. The resulting stats.engineBreakdown lists techniques like rtk-render:terraformPlan, rtk-truncate, and caveman-noise-drop.

Caveman-Only Lite Mode

{
  "engines": {
    "caveman": {
      "enabled": true,
      "intensity": "lite"
    }
  }
}

Bypasses RTK entirely. Caveman removes obvious boilerplate and caps line lengths, typically achieving 5-10% token savings with negligible latency.

RTK-Only with Standard Intensity

{
  "engines": {
    "rtk": { 
      "enabled": true, 
      "intensity": "standard" 
    },
    "caveman": { 
      "enabled": false 
    }
  }
}

Useful when you need RTK's semantic renderers (e.g., converting verbose git diff output into summaries) without Caveman's aggressive truncation.

Key Source Files

Purpose	File Path
RTK engine core	open-sse/services/compression/engines/rtk/index.ts
RTK configuration schema	open-sse/services/compression/engines/rtk/configSchema.ts
Caveman engine	open-sse/services/compression/engines/cavemanAdapter.ts
Pipeline construction	open-sse/services/compression/strategySelector.ts
Statistics aggregation	open-sse/services/compression/stats.ts
Request normalization	open-sse/services/compression/bodyAdapter.ts

Summary

• Prompt compression in OmniRoute occurs before provider transmission through a modular pipeline defined in strategySelector.ts.
• RTK (priority 10) provides semantic, command-aware compression using declarative filters and optional renderers for specific output types like git diff or terraform plan.
• Caveman (priority 20) applies aggressive, rule-based heuristics after RTK to remove noise and enforce line budgets.
• The stacking order ensures semantic processing precedes brute-force truncation, maximizing token savings while preserving critical information.
• Both engines report detailed statistics via createCompressionStats(), including techniques used and raw-output retention pointers.

Frequently Asked Questions

What is the difference between RTK and Caveman compression?

RTK uses declarative filters and semantic renderers to understand command output types (like Docker logs or Git diffs) and compress them intelligently, while Caveman applies aggressive, regex-based rules to remove noise and trim lines without context awareness. RTK runs first to preserve semantic structure, and Caveman follows to maximize token reduction.

How do I configure both engines to work together?

Set both engines.rtk.enabled and engines.caveman.enabled to true in your configuration JSON. The strategySelector.ts module automatically orders them by stackPriority (RTK at 10, Caveman at 20), ensuring RTK processes the content first and Caveman receives the intermediate result.

Can I use Caveman without RTK?

Yes. Set engines.rtk.enabled to false and engines.caveman.enabled to true. In this mode, requests bypass all RTK-specific logic including semantic renderers and deduplication, relying solely on Caveman's rule-based heuristics for compression.

How does OmniRoute handle compression statistics?

The createCompressionStats() function in open-sse/services/compression/stats.ts aggregates results from each engine in the pipeline, recording original token counts, compressed token counts, specific techniques applied (e.g., rtk-filter, caveman-noise-drop), and pointers to retained raw outputs for auditing.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}