# How RTK and Caveman Compression Save Tokens in OmniRoute

> Discover how OmniRoute uses RTK and Caveman compression to slash LLM token usage by up to 95%. Learn about cascaded filters, deduplication, and truncation techniques.

- Repository: [Diego Rodrigues de Sa e Souza/OmniRoute](https://github.com/diegosouzapw/OmniRoute)
- Tags: performance
- Published: 2026-08-10

---

**TLDR:** OmniRoute reduces LLM token consumption by approximately **78%–95%** by chaining the **RTK** (Run-Time-Kit) engine for command-aware compression with the **Caveman** engine for prose-aware compression, applying cascaded filters, deduplication, and intensity-based truncation to tool outputs before they reach the model.

OmniRoute is an open-source routing layer that dramatically reduces LLM costs by intercepting and compressing noisy machine-generated outputs before they enter the context window. By combining two specialized compression engines—RTK for structured command output and Caveman for narrative text—the system achieves multiplicative token savings while preserving critical error information and semantic intent. This article explains how RTK and Caveman compression save tokens based on the actual implementation in the `diegosouzapw/OmniRoute` repository.

## Understanding the RTK Compression Engine

The **RTK** (Run-Time-Kit) engine provides **command-aware** compression that targets noisy machine-generated output from tools like `git`, `npm test`, Docker, and build runners. According to the RTK benchmark data cited in [`docs/compression/RTK_COMPRESSION.md`](https://github.com/diegosouzapw/OmniRoute/blob/main/docs/compression/RTK_COMPRESSION.md), this engine alone achieves **60%–90%** token reduction, averaging approximately **80%** savings on typical CI/CD and development output.

### Command Detection and Filter Loading

In [`open-sse/services/compression/engines/rtk/commandDetector.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/engines/rtk/commandDetector.ts), the system classifies incoming tool output to determine the appropriate compression strategy. The engine then loads filters in a specific hierarchy:

1. Project-level filters from `.rtk/filters.*`
2. Global filters from `DATA_DIR/rtk/filters.*`
3. Built-in filters from `open-sse/services/compression/engines/rtk/filters/`

Each filter follows the JSON schema defined in [`open-sse/services/compression/engines/rtk/filterSchema.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/engines/rtk/filterSchema.ts), specifying rules for stripping ANSI codes, pattern matching, inclusion/exclusion logic, and truncation.

### Filter DSL and Deduplication

The RTK engine applies a **cascade of filters** that perform text transformations without sending raw output to the LLM (unless `rawOutputRetention` is enabled for debugging). Key mechanisms include:

- **Rule-based deduplication**: Individual filters can specify `rules.deduplicate` to collapse repeated lines within their scope.
- **Engine-wide deduplication**: A final pass in [`open-sse/services/compression/engines/rtk/index.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/engines/rtk/index.ts) collapses runs of identical consecutive lines using `deduplicateThreshold` (default: **3** lines).
- **Similarity grouping**: When `rtkConfig.enableGrouping` is true, the `groupSimilarLines` function collapses near-identical consecutive lines into single representative entries.

### Intensity Levels and Truncation

RTK supports three **intensity levels** that control truncation aggressiveness:

- **minimal**: Preserves most content with light truncation
- **standard** (default): Truncates sections to approximately **24 lines** per section
- **aggressive**: Truncates sections to approximately **16 lines** per section, maximizing token savings for high-volume outputs

## Understanding the Caveman Compression Engine

**Caveman** serves as the second-stage **prose-aware** compressor that processes RTK-completed output to remove narrative noise. Located in [`open-sse/services/compression/caveman.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/caveman.ts), this engine applies heuristics specifically tuned for LLM-friendly summarization, achieving an additional **~46%** reduction on the already-compressed RTK output.

### Prose-Aware Heuristics

The compression rules defined in [`open-sse/services/compression/cavemanRules.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/cavemanRules.ts) perform several text optimization strategies:

- **Hedging removal**: Strips filler phrases (e.g., "as far as I can tell") that add token overhead without semantic value
- **Structural pruning**: Collapses repetitive stack traces and condenses long log sequences
- **Error preservation**: Maintains critical error messages, stack traces, and test failure details essential for LLM reasoning

### Secondary Deduplication

After applying per-rule filters, Caveman runs a final deduplication pass using the same threshold (**3 identical consecutive lines**) as the RTK engine. This ensures that any repetitive patterns surviving the first stage are eliminated before the content enters the LLM context window.

## Calculating Stacked Token Savings

When the two engines are chained (`rtk → caveman`), the combined effect is multiplicative rather than additive:

```

Stacked saving = 1 – (1 – RTK_save) × (1 – Caveman_save)
               = 1 – (1 – 0.80) × (1 – 0.46)
               ≈ 89% overall token reduction

```

Based on the RTK variability range (60%–90%), the **overall token savings** fall between **78% and 95%**.

| Engine | Compression Strategy | Typical Savings |
|--------|---------------------|-----------------|
| **RTK** | Command-aware filtering and truncation | **~80%** (60%–90% range) |
| **Caveman** | Prose-aware summarization and hedging removal | **~46%** of RTK output |
| **Combined** | Stacked pipeline | **~89%** (78%–95% range) |

## Configuring Compression Pipelines

You can interact with these engines via the REST API or programmatic configuration.

### Preview Stacked Compression

To test the combined pipeline before deployment:

```json
POST /api/compression/preview
{
  "mode": "stacked",
  "messages": [
    {
      "role": "tool",
      "content": "FAIL tests/example.test.ts\nAssertionError: expected true\nTest Files 1 failed"
    }
  ],
  "config": {
    "rtkConfig": { "intensity": "standard" },
    "cavemanConfig": { "intensity": "full" }
  }
}

```

The response includes the compressed payload and a `CompressionStats` object detailing individual savings from each engine.

### Adjusting Intensity for Maximum Savings

For high-volume environments, configure aggressive compression:

```typescript
import { updateEngineConfig } from "@omniroute/open-sse/services/compression/engines/registry";

updateEngineConfig("rtk", { intensity: "aggressive" });
updateEngineConfig("caveman", { intensity: "full" });

```

Setting RTK to **aggressive** reduces section limits to 16 lines, while **full** Caveman intensity applies all available heuristics, pushing total reduction toward the **~95%** upper bound.

### Retaining Raw Output for Debugging

To preserve uncompressed copies for audit purposes:

```json
PUT /api/context/rtk/config
{
  "rawOutputRetention": "always",
  "maxCharsPerResult": 12000
}

```

Raw outputs are stored in `DATA_DIR/rtk/raw-output/` and accessible via:

```bash
GET /api/context/rtk/raw-output/<id>

```

## Summary

- **RTK compression** (in [`open-sse/services/compression/engines/rtk/index.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/engines/rtk/index.ts)) provides command-aware filtering that reduces tokens by **60%–90%** through pattern matching, deduplication, and configurable intensity truncation.
- **Caveman compression** (in [`open-sse/services/compression/caveman.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/caveman.ts)) applies prose-aware heuristics that further reduce the RTK-compressed payload by approximately **46%**.
- **Stacked configuration** achieves **78%–95%** total token reduction by multiplicatively combining both engines.
- **Configuration persistence** is handled in [`src/lib/db/compression.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/db/compression.ts) using an SQLite `key_value` table.
- **Quality assurance** is maintained through the golden set tests in [`tests/golden-set/compression-caveman-v2.test.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/tests/golden-set/compression-caveman-v2.test.ts).

## Frequently Asked Questions

### What is the typical token reduction when using both RTK and Caveman compression?

When chained together, RTK and Caveman compression save approximately **89%** of tokens on average, with a range between **78%** and **95%** depending on the input noise level and selected intensity settings. RTK handles the initial heavy lifting on structured command output, while Caveman optimizes the remaining prose.

### How does the RTK engine determine which filters to apply?

The [`commandDetector.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/commandDetector.ts) module classifies incoming tool output (e.g., `git diff`, `npm test`) and loads appropriate filters in order: project-level (`.rtk/filters.*`), global (`DATA_DIR/rtk/filters.*`), and finally built-in defaults. Each filter is validated against the schema in [`filterSchema.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/filterSchema.ts) before application.

### What is the difference between RTK and Caveman compression?

**RTK** is **command-aware**, designed for structured machine output like logs and diffs, using pattern matching and line-based deduplication. **Caveman** is **prose-aware**, designed for narrative text, using heuristics to remove hedging language and collapse repetitive phrases while preserving critical error context.

### Can I view the original uncompressed output after RTK compression?

Yes. Setting `rawOutputRetention` to `"always"` in the RTK configuration (stored via [`src/lib/db/compression.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/db/compression.ts)) preserves redacted originals in `DATA_DIR/rtk/raw-output/`. You can retrieve these using the `GET /api/context/rtk/raw-output/<id>` endpoint for debugging or verification against the compressed results.