# How RTK + Caveman Compression Saves 15-95% Tokens in OmniRoute

> Discover how RTK + Caveman compression in OmniRoute drastically cuts tokens by 15-95%. Learn about the stacked pipeline that preserves meaning while optimizing output.

- Repository: [Diego Rodrigues de Sa e Souza/OmniRoute](https://github.com/diegosouzapw/OmniRoute)
- Tags: internals
- Published: 2026-08-07

---

**OmniRoute implements RTK + Caveman compression as a stacked two-stage pipeline where RTK (priority 10) trims tool outputs and Caveman (priority 20) cleans prose, together cutting approximately 9% of tokens while preserving semantic meaning.**

The **RTK + Caveman compression** system in [OmniRoute](https://github.com/diegosouzapw/OmniRoute) is designed to reduce LLM request payloads before they reach upstream providers. This article explains exactly how the two engines coordinate, where the token savings come from, and how to configure the pipeline in your own deployment.

## How the Stacked Pipeline Works

OmniRoute's compression architecture relies on **priority-ordered engines** that transform the request body sequentially. The default configuration runs RTK first, then Caveman, though the system supports custom pipelines.

### Pipeline Selection and Execution Order

In [`open-sse/services/compression/strategySelector.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/strategySelector.ts), the orchestration logic builds the engine stack:

```ts
// Default fallback pipeline: RTK → Caveman
const defaultPipeline = [rtkEngine, cavemanEngine];

```

The priority values (10 for RTK, 20 for Caveman) ensure deterministic execution order. Higher numbers run later, so Caveman receives already-trimmed output from RTK.

### Stage 1: RTK Engine (Tool Output Trimming)

The **RTK** (Run-Time-Kit) engine in [`open-sse/services/compression/engines/rtk/index.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/engines/rtk/index.ts) focuses on **command-aware filtering of tool results**. Its entry point `applyRtkCompression(body, options)` performs:

1. **Tool-call resolution** – Scans chat history to map `tool_call_id` to `{toolName, command}` for both OpenAI and Anthropic message formats
2. **Message filtering** – Applies filters only to relevant content (shell commands trigger filtering; non-shell tools may skip)
3. **Transform pipeline** – Executes head/tail line trimming, regex replacement, deduplication, grouping, and truncation based on `intensity` (`minimal` | `standard` | `aggressive`)
4. **Stats recording** – Tracks original vs. compressed token counts and techniques used

**Key source:** `processRtkContent` scales line budgets via `effectiveMaxLines` and supports **raw-output retention** when `rtkRawOutputRetention` is enabled, storing originals to `DATA_DIR/rtk/raw-output/*.log`.

### Stage 2: Caveman Engine (Prose Cleanup)

The **Caveman** engine in [`open-sse/services/compression/caveman.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/caveman.ts) performs **deterministic rule-based text compression** through `cavemanCompress(text, config)`:

1. **Block preservation** – `extractPreservedBlocks` identifies code fences and protected identifiers, removing them before rule application
2. **Rule application** – `applyRulesToText` runs ordered regex rules from [`cavemanRules.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/cavemanRules.ts) (filler adverbs, redundant phrasing, pleasantries)
3. **Keyword-guarded execution** – `shouldAttemptRule` skips rules whose keywords aren't present, keeping passes fast
4. **Whitespace normalization** – Collapses horizontal runs, strips spaces before punctuation, and removes excess newlines
5. **Recapitalization** – `recapitalizeSentences` restores proper casing after aggressive cleanup

**Key source:** `getRulesForContext` selects rule subsets based on intensity, and `estimateCompressionTokens` provides token savings estimates.

### Result Aggregation

The [`open-sse/services/compression/stackedStepCore.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/stackedStepCore.ts) module merges per-engine statistics into a final `CompressionResult`:

- Sets `compressed: true` when `compressedTokens < originalTokens`
- Aggregates `techniquesUsed` from both engines
- Preserves the transformed request body for upstream delivery

## Why RTK + Caveman Saves 15-95% Tokens

The **RTK + Caveman compression** savings come from complementary attack surfaces:

| Engine | Primary Target | Typical Savings | Max Savings |
|--------|---------------|---------------|-------------|
| **RTK** | Large tool outputs (logs, diffs, CLI results) | 30-80% of tool content | 95% on verbose commands |
| **Caveman** | Chat prose, filler, whitespace | 5-12% of remaining tokens | 15% on verbose conversations |
| **Combined** | Full request payload | ≈ 9% average | 15-95% range |

**RTK dominates** when requests contain substantial command output. A 600-token `git log` or `kubectl describe` result trimmed to 60 tokens via head/tail filtering and deduplication represents a **90% reduction** of that block. **Caveman captures** the residual inefficiency in conversational text—phrases like "I'd be happy to help you with that" compress to "I'd be happy to help" through `filler_adverbs` and `redundant_phrasing` rules.

The observed **≈ 9% overall figure** reflects production traffic mixing tool-heavy requests (high savings) with simple conversational turns (minimal savings). Pure tool-result payloads routinely achieve **50-95% compression**.

## Implementing RTK + Caveman in Your Code

### Default Pipeline: Automatic Stacked Execution

```ts
import { applyCompressionPipeline } from '@/open-sse/services/compression/index.ts';

const body = {
  messages: [
    { role: 'user', content: 'Show me the last 20 commits.' },
    // Assistant tool_call and tool result messages follow...
  ],
};

const result = await applyCompressionPipeline(body);

console.log({
  compressed: result.compressed,
  saved: result.stats?.originalTokens - result.stats?.compressedTokens,
  techniques: result.stats?.techniquesUsed,
});

```

`applyCompressionPipeline` internally chains `applyRtkCompression` → `cavemanCompress` through [`stackedStepCore.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/stackedStepCore.ts).

### RTK-Only Configuration

```ts
import { applyRtkCompression } from '@/open-sse/services/compression/engines/rtk/index.ts';

const rtkResult = applyRtkCompression(body, {
  config: { 
    enabled: true, 
    intensity: 'aggressive',
    maxLines: 20,
    dedupEnabled: true 
  },
});

// Access per-engine stats
const savings = rtkResult.stats?.originalTokens - rtkResult.stats?.compressedTokens;

```

**Key configuration options:** `intensity` scales line limits, `dedupEnabled` triggers repeated-line removal, and `rawOutputRetention` preserves originals for debugging.

### Caveman-Only for Prose Cleanup

```ts
import { cavemanCompress } from '@/open-sse/services/compression/caveman.ts';

const verbose = `
  Thanks so much for your patience!  I would really appreciate it if you could 
  please check the configuration file first.  Also, please make sure you have 
  the latest version installed.
`;

const { text, stats } = cavemanCompress(verbose);

console.log(text); // "Thanks for your patience! I would appreciate it if you could check the configuration file first. Also make sure you have the latest version installed."
console.log(stats?.compressedTokens); // Reduced token count

```

## Key Source Files and Architecture

| File | Responsibility |
|------|--------------|
| [[`open-sse/services/compression/strategySelector.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/strategySelector.ts)](https://github.com/diegosouzapw/OmniRoute/blob/release/v3.8.50/open-sse/services/compression/strategySelector.ts) | Pipeline assembly, engine priority resolution |
| [[`open-sse/services/compression/engines/rtk/index.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/engines/rtk/index.ts)](https://github.com/diegosouzapw/OmniRoute/blob/release/v3.8.50/open-sse/services/compression/engines/rtk/index.ts) | RTK implementation, tool-call mapping, filter execution |
| [[`open-sse/services/compression/engines/cavemanAdapter.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/engines/cavemanAdapter.ts)](https://github.com/diegosouzapw/OmniRoute/blob/release/v3.8.50/open-sse/services/compression/engines/cavemanAdapter.ts) | Caveman wrapper conforming to `CompressionEngine` interface |
| [[`open-sse/services/compression/caveman.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/caveman.ts)](https://github.com/diegosouzapw/OmniRoute/blob/release/v3.8.50/open-sse/services/compression/caveman.ts) | Rule-based text compression, whitespace cleanup |
| [[`open-sse/services/compression/stackedStepCore.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/stackedStepCore.ts)](https://github.com/diegosouzapw/OmniRoute/blob/release/v3.8.50/open-sse/services/compression/stackedStepCore.ts) | Multi-engine orchestration, stats aggregation |
| [[`open-sse/services/compression/types.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/types.ts)](https://github.com/diegosouzapw/OmniRoute/blob/release/v3.8.50/open-sse/services/compression/types.ts) | Shared types: `CompressionResult`, `CompressionEngine`, `CompressionStats` |

## Summary

- **RTK + Caveman compression** in OmniRoute uses a **priority-stacked pipeline** (RTK priority 10, Caveman priority 20) to transform request payloads before upstream delivery
- **RTK** achieves **30-80% compression** on tool outputs through command-aware filtering, line limits, deduplication, and truncation
- **Caveman** adds **5-12% additional savings** via deterministic prose rules that remove filler, collapse whitespace, and normalize punctuation
- Combined savings average **≈ 9%** across production traffic, with **15-95% possible** depending on payload composition
- Configure via `applyCompressionPipeline` for automatic stacking, or invoke `applyRtkCompression` and `cavemanCompress` independently for custom pipelines

## Frequently Asked Questions

### What does RTK stand for in OmniRoute compression?

**RTK** stands for **Run-Time-Kit**. It is the first-stage compression engine that operates on tool execution results, applying command-specific filters to trim verbose output like logs and diffs before they reach the LLM.

### How does Caveman compression preserve code blocks while cleaning prose?

Caveman uses `extractPreservedBlocks` to identify and remove code fences, JSON blobs, and protected identifiers before applying regex rules. After rule execution, `restorePreservedBlocks` re-inserts the original protected content unchanged. This ensures deterministic compression without corrupting executable code or structured data.

### Can I disable one engine and use only RTK or only Caveman?

Yes. The [`strategySelector.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/strategySelector.ts) module supports custom engine arrays. Pass a single-element pipeline to `applyCompressionPipeline`, or import and call `applyRtkCompression` or `cavemanCompress` directly as shown in the implementation examples above.

### Why is the average savings only 9% when individual engines claim higher percentages?

The **≈ 9% figure** represents production averages across all request types. Conversational turns with minimal tool output see small savings (0-5%), while tool-heavy requests achieve 50-95%. The average weights these by traffic volume. For workloads dominated by CLI or API tool results, expect significantly higher compression ratios.