How RTK + Caveman Compression Saves 15-95% Tokens in OmniRoute

OmniRoute implements RTK + Caveman compression as a stacked two-stage pipeline where RTK (priority 10) trims tool outputs and Caveman (priority 20) cleans prose, together cutting approximately 9% of tokens while preserving semantic meaning.

The RTK + Caveman compression system in OmniRoute is designed to reduce LLM request payloads before they reach upstream providers. This article explains exactly how the two engines coordinate, where the token savings come from, and how to configure the pipeline in your own deployment.

How the Stacked Pipeline Works

OmniRoute's compression architecture relies on priority-ordered engines that transform the request body sequentially. The default configuration runs RTK first, then Caveman, though the system supports custom pipelines.

Pipeline Selection and Execution Order

In open-sse/services/compression/strategySelector.ts, the orchestration logic builds the engine stack:

// Default fallback pipeline: RTK → Caveman
const defaultPipeline = [rtkEngine, cavemanEngine];

The priority values (10 for RTK, 20 for Caveman) ensure deterministic execution order. Higher numbers run later, so Caveman receives already-trimmed output from RTK.

Stage 1: RTK Engine (Tool Output Trimming)

The RTK (Run-Time-Kit) engine in open-sse/services/compression/engines/rtk/index.ts focuses on command-aware filtering of tool results. Its entry point applyRtkCompression(body, options) performs:

1. Tool-call resolution – Scans chat history to map tool_call_id to {toolName, command} for both OpenAI and Anthropic message formats
2. Message filtering – Applies filters only to relevant content (shell commands trigger filtering; non-shell tools may skip)
3. Transform pipeline – Executes head/tail line trimming, regex replacement, deduplication, grouping, and truncation based on intensity (minimal | standard | aggressive)
4. Stats recording – Tracks original vs. compressed token counts and techniques used

Key source: processRtkContent scales line budgets via effectiveMaxLines and supports raw-output retention when rtkRawOutputRetention is enabled, storing originals to DATA_DIR/rtk/raw-output/*.log.

Stage 2: Caveman Engine (Prose Cleanup)

The Caveman engine in open-sse/services/compression/caveman.ts performs deterministic rule-based text compression through cavemanCompress(text, config):

1. Block preservation – extractPreservedBlocks identifies code fences and protected identifiers, removing them before rule application
2. Rule application – applyRulesToText runs ordered regex rules from cavemanRules.ts (filler adverbs, redundant phrasing, pleasantries)
3. Keyword-guarded execution – shouldAttemptRule skips rules whose keywords aren't present, keeping passes fast
4. Whitespace normalization – Collapses horizontal runs, strips spaces before punctuation, and removes excess newlines
5. Recapitalization – recapitalizeSentences restores proper casing after aggressive cleanup

Key source: getRulesForContext selects rule subsets based on intensity, and estimateCompressionTokens provides token savings estimates.

Result Aggregation

The open-sse/services/compression/stackedStepCore.ts module merges per-engine statistics into a final CompressionResult:

• Sets compressed: true when compressedTokens < originalTokens
• Aggregates techniquesUsed from both engines
• Preserves the transformed request body for upstream delivery

Why RTK + Caveman Saves 15-95% Tokens

The RTK + Caveman compression savings come from complementary attack surfaces:

Engine	Primary Target	Typical Savings	Max Savings
RTK	Large tool outputs (logs, diffs, CLI results)	30-80% of tool content	95% on verbose commands
Caveman	Chat prose, filler, whitespace	5-12% of remaining tokens	15% on verbose conversations
Combined	Full request payload	≈ 9% average	15-95% range

RTK dominates when requests contain substantial command output. A 600-token git log or kubectl describe result trimmed to 60 tokens via head/tail filtering and deduplication represents a 90% reduction of that block. Caveman captures the residual inefficiency in conversational text—phrases like "I'd be happy to help you with that" compress to "I'd be happy to help" through filler_adverbs and redundant_phrasing rules.

The observed ≈ 9% overall figure reflects production traffic mixing tool-heavy requests (high savings) with simple conversational turns (minimal savings). Pure tool-result payloads routinely achieve 50-95% compression.

Implementing RTK + Caveman in Your Code

Default Pipeline: Automatic Stacked Execution

import { applyCompressionPipeline } from '@/open-sse/services/compression/index.ts';

const body = {
  messages: [
    { role: 'user', content: 'Show me the last 20 commits.' },
    // Assistant tool_call and tool result messages follow...
  ],
};

const result = await applyCompressionPipeline(body);

console.log({
  compressed: result.compressed,
  saved: result.stats?.originalTokens - result.stats?.compressedTokens,
  techniques: result.stats?.techniquesUsed,
});

applyCompressionPipeline internally chains applyRtkCompression → cavemanCompress through stackedStepCore.ts.

RTK-Only Configuration

import { applyRtkCompression } from '@/open-sse/services/compression/engines/rtk/index.ts';

const rtkResult = applyRtkCompression(body, {
  config: { 
    enabled: true, 
    intensity: 'aggressive',
    maxLines: 20,
    dedupEnabled: true 
  },
});

// Access per-engine stats
const savings = rtkResult.stats?.originalTokens - rtkResult.stats?.compressedTokens;

Key configuration options: intensity scales line limits, dedupEnabled triggers repeated-line removal, and rawOutputRetention preserves originals for debugging.

Caveman-Only for Prose Cleanup

import { cavemanCompress } from '@/open-sse/services/compression/caveman.ts';

const verbose = `
  Thanks so much for your patience!  I would really appreciate it if you could 
  please check the configuration file first.  Also, please make sure you have 
  the latest version installed.
`;

const { text, stats } = cavemanCompress(verbose);

console.log(text); // "Thanks for your patience! I would appreciate it if you could check the configuration file first. Also make sure you have the latest version installed."
console.log(stats?.compressedTokens); // Reduced token count

Key Source Files and Architecture

File	Responsibility
[open-sse/services/compression/strategySelector.ts](https://github.com/diegosouzapw/OmniRoute/blob/release/v3.8.50/open-sse/services/compression/strategySelector.ts)	Pipeline assembly, engine priority resolution
[open-sse/services/compression/engines/rtk/index.ts](https://github.com/diegosouzapw/OmniRoute/blob/release/v3.8.50/open-sse/services/compression/engines/rtk/index.ts)	RTK implementation, tool-call mapping, filter execution
[open-sse/services/compression/engines/cavemanAdapter.ts](https://github.com/diegosouzapw/OmniRoute/blob/release/v3.8.50/open-sse/services/compression/engines/cavemanAdapter.ts)	Caveman wrapper conforming to CompressionEngine interface
[open-sse/services/compression/caveman.ts](https://github.com/diegosouzapw/OmniRoute/blob/release/v3.8.50/open-sse/services/compression/caveman.ts)	Rule-based text compression, whitespace cleanup
[open-sse/services/compression/stackedStepCore.ts](https://github.com/diegosouzapw/OmniRoute/blob/release/v3.8.50/open-sse/services/compression/stackedStepCore.ts)	Multi-engine orchestration, stats aggregation
[open-sse/services/compression/types.ts](https://github.com/diegosouzapw/OmniRoute/blob/release/v3.8.50/open-sse/services/compression/types.ts)	Shared types: CompressionResult, CompressionEngine, CompressionStats

Summary

• RTK + Caveman compression in OmniRoute uses a priority-stacked pipeline (RTK priority 10, Caveman priority 20) to transform request payloads before upstream delivery
• RTK achieves 30-80% compression on tool outputs through command-aware filtering, line limits, deduplication, and truncation
• Caveman adds 5-12% additional savings via deterministic prose rules that remove filler, collapse whitespace, and normalize punctuation
• Combined savings average ≈ 9% across production traffic, with 15-95% possible depending on payload composition
• Configure via applyCompressionPipeline for automatic stacking, or invoke applyRtkCompression and cavemanCompress independently for custom pipelines

Frequently Asked Questions

What does RTK stand for in OmniRoute compression?

RTK stands for Run-Time-Kit. It is the first-stage compression engine that operates on tool execution results, applying command-specific filters to trim verbose output like logs and diffs before they reach the LLM.

How does Caveman compression preserve code blocks while cleaning prose?

Caveman uses extractPreservedBlocks to identify and remove code fences, JSON blobs, and protected identifiers before applying regex rules. After rule execution, restorePreservedBlocks re-inserts the original protected content unchanged. This ensures deterministic compression without corrupting executable code or structured data.

Can I disable one engine and use only RTK or only Caveman?

Yes. The strategySelector.ts module supports custom engine arrays. Pass a single-element pipeline to applyCompressionPipeline, or import and call applyRtkCompression or cavemanCompress directly as shown in the implementation examples above.

Why is the average savings only 9% when individual engines claim higher percentages?

The ≈ 9% figure represents production averages across all request types. Conversational turns with minimal tool output see small savings (0-5%), while tool-heavy requests achieve 50-95%. The average weights these by traffic volume. For workloads dominated by CLI or API tool results, expect significantly higher compression ratios.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}