# How OmniRoute Implements RTK + Caveman Prompt Compression to Save Tokens

> Discover how OmniRoute's RTK + Caveman prompt compression pipeline cuts LLM token costs by removing redundant content while preserving meaning.

- Repository: [Diego Rodrigues de Sa e Souza/OmniRoute](https://github.com/diegosouzapw/OmniRoute)
- Tags: internals
- Published: 2026-07-12

---

**OmniRoute reduces upstream LLM token costs by running requests through a stackable compression pipeline that combines the RTK (Rapid-Tool-Kit) engine with the Caveman engine to strip redundant content while preserving semantic meaning.**

The OmniRoute repository implements a sophisticated **prompt compression** system that intercepts request bodies before they reach expensive LLM providers. By default, every request passes through a sequential pipeline where the RTK engine handles tool-output compression, followed by the Caveman engine applying rule-based text transformations. This dual-stage approach can reduce token counts by over 30% in multi-tool conversations.

## The Compression Pipeline Architecture

The pipeline is assembled by **[`strategySelector.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/strategySelector.ts)**, which reads the combo configuration and creates an ordered list of engine specifications based on their **stackPriority** values. Engines marked as `stackable: true` execute sequentially, with RTK assigned priority `10` and Caveman assigned priority `20`, resulting in the default stacked pipeline `[rtk, caveman]`【/cache/repos/github.com/diegosouzapw/OmniRoute/main/open-sse/services/compression/strategySelector.ts#L138-L150】.

This ordering ensures that command-aware filtering happens first, followed by general text compression rules that operate on the already-reduced content.

## RTK Engine: Command-Aware Tool Output Compression

The RTK engine, implemented in **[`rtk/index.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/rtk/index.ts)**, specializes in compressing tool execution outputs and code blocks while preserving error lines and semantic structure.

### Entry Point and Message Filtering

The `applyRtkCompression` function serves as the entry point, receiving the request body and a configuration object【/cache/repos/github.com/diegosouzapw/OmniRoute/main/open-sse/services/compression/engines/rtk/index.ts#L25-L38】. For each message, `shouldCompressMessage` evaluates whether compression applies based on role types, tool-result flags, and the presence of code fences【/cache/repos/github.com/diegosouzapw/OmniRoute/main/open-sse/services/compression/engines/rtk/index.ts#L42-L46】.

### Tool Metadata and Command Detection

Before processing, RTK builds a **tool-metadata lookup** from preceding assistant messages to determine whether a tool result represents a shell command (which receives filtering) or a non-shell operation (which skips filters)【/cache/repos/github.com/diegosouzapw/OmniRoute/main/open-sse/services/compression/engines/rtk/index.ts#L49-L66】. The `detectCommandType` function classifies the command type, while `matchRtkFilter` selects appropriate declarative filters unless the output resembles a "document-like read" that requires preservation【/cache/repos/github.com/diegosouzapw/OmniRoute/main/open-sse/services/compression/engines/rtk/index.ts#L40-L45】.

### Core Processing Pipeline

The `processRtkText` function executes a six-step transformation:

1. **Command Detection**: Identifies the command type using `detectCommandType`.
2. **Filter Matching**: Applies declarative filters via `matchRtkFilter`, unless the output is document-like.
3. **Rendering and Code Stripping**: Optionally applies `applyRenderer` and strips fenced code blocks.
4. **Deduplication**: Removes repeated lines via `deduplicateRepeatedLines` and optionally groups similar lines via `groupSimilarLines`.
5. **Smart Truncation**: Truncates based on `effectiveMaxLines` intensity while preserving error lines【/cache/repos/github.com/diegosouzapw/OmniRoute/main/open-sse/services/compression/engines/rtk/index.ts#L28-L34】.
6. **Raw Output Retention**: Calls `maybePersistRtkRawOutput` to keep raw pointers when compressed text is shorter, enabling later inspection.

The engine reports its work through `createCompressionStats`, which tracks techniques used (such as `rtk-filter` and `rtk-raw-output-retention`) and calculates token estimates via `estimateCompressionTokens`【/cache/repos/github.com/diegosouzapw/OmniRoute/main/open-sse/services/compression/engines/rtk/index.ts#L44-L48】.

## Caveman Engine: Rule-Based Message Compression

Following RTK, the Caveman engine applies language-aware text rules to further reduce token count. The engine is implemented in **[`cavemanAdapter.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/cavemanAdapter.ts)** and exported as `cavemanEngine` with `stackPriority: 20`【/cache/repos/github.com/diegosouzapw/OmniRoute/main/open-sse/services/compression/engines/cavemanAdapter.ts#L73-L81】.

### Configuration and Adaptation

The `apply` method first adapts the request body via `adaptBodyForCompression`, then constructs a **caveman configuration** that merges user-provided settings, step-level overrides, and language-pack configurations【/cache/repos/github.com/diegosouzapw/OmniRoute/main/open-sse/services/compression/engines/cavemanAdapter.ts#L92-L108】. Notably, the wrapper automatically enables the engine (`{ enabled: true }`) when no explicit flag exists in the user configuration.

### Text Transformation Rules

The actual compression delegates to **`cavemanCompress`** (located in [`caveman.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/caveman.ts)), which executes text-rules loaded from [`cavemanRules.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/cavemanRules.ts). These rules perform operations including:

- Removing redundant system-prompt tokens
- Collapsing whitespace and duplicate lines
- Preserving user-specified language packs
- Stripping non-essential formatting

Because Caveman runs after RTK in the stacked pipeline, it operates on already-filtered content, achieving cumulative compression rather than redundant processing.

## Tracking Token Savings and Statistics

After each engine completes, **`createCompressionStats`** (in [`stats.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/stats.ts)) records the `originalTokens`, `compressedTokens`, list of `techniquesUsed`, and per-engine breakdowns【/cache/repos/github.com/diegosouzapw/OmniRoute/main/open-sse/services/compression/stats.ts#L1-L30】. The final response includes a `stats` object where `compressedTokens < originalTokens` confirms successful token reduction.

This telemetry enables developers to verify that **prompt compression** is actively reducing costs and to identify which specific techniques (RTK filtering versus Caveman rules) contributed most to the savings.

## Configuring the Compression Pipeline

You can interact with the compression system via the API. The following example demonstrates how requests automatically flow through the default RTK → Caveman pipeline:

```typescript
import { fetch } from 'node-fetch';

const body = {
  model: 'gpt-4o',
  messages: [
    { role: 'user', content: 'Show me the logs of my recent build.' },
    // Tool results and code blocks follow...
  ],
  // Optional: Override the default pipeline order
  // pipeline: ['caveman', 'rtk'],
};

await fetch('https://my.omniroute.dev/api/v1/chat/completions', {
  method: 'POST',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify(body),
})
  .then(r => r.json())
  .then(res => {
    console.log('Compressed?', res.stats.compressed);
    console.log('Original tokens:', res.stats.originalTokens);
    console.log('Compressed tokens:', res.stats.compressedTokens);
    console.log('Techniques used:', res.stats.techniquesUsed);
  });

```

The request automatically passes through [`strategySelector.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/strategySelector.ts), which assembles the stacked pipeline `[rtk, caveman]` based on the default priorities. The response includes detailed statistics showing exactly how many tokens were saved and which compression techniques were applied.

## Summary

- **Stackable Pipeline**: OmniRoute uses [`strategySelector.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/strategySelector.ts) to assemble a prioritized pipeline where RTK (priority 10) runs before Caveman (priority 20), ensuring command-aware filtering precedes general text compression.
- **RTK Engine**: Located in [`rtk/index.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/rtk/index.ts), this engine uses `detectCommandType`, `matchRtkFilter`, and `processRtkText` to deduplicate lines, truncate low-value content, and preserve error lines in tool outputs.
- **Caveman Engine**: Implemented in [`cavemanAdapter.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/cavemanAdapter.ts) and [`caveman.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/caveman.ts), this engine applies rule-based transformations including whitespace collapse and system-prompt reduction.
- **Token Tracking**: The `createCompressionStats` function in [`stats.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/stats.ts) records original versus compressed token counts and lists specific techniques applied by each engine.
- **Cumulative Savings**: Running both engines sequentially typically achieves 30%+ token reduction by removing repetitive log lines, fenced code comments, and redundant whitespace before the request reaches the LLM provider.

## Frequently Asked Questions

### How does the RTK engine decide which messages to compress?

The RTK engine uses the `shouldCompressMessage` function in [`rtk/index.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/rtk/index.ts) to evaluate each message based on role type, tool-result flags, and the presence of code fences【/cache/repos/github.com/diegosouzapw/OmniRoute/main/open-sse/services/compression/engines/rtk/index.ts#L42-L46】. It specifically targets tool outputs and code blocks while preserving user instructions and system prompts that require full context.

### Can I change the order of the compression engines?

Yes, the pipeline order is configurable through the [`strategySelector.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/strategySelector.ts) configuration. While the default stacked pipeline is `[rtk, caveman]` based on their `stackPriority` values (10 and 20 respectively), you can override this by specifying a custom pipeline array in your request, such as `['caveman', 'rtk']`【/cache/repos/github.com/diegosouzapw/OmniRoute/main/open-sse/services/compression/strategySelector.ts#L138-L150】.

### What happens if the compressed text ends up longer than the original?

The RTK engine includes a `maybePersistRtkRawOutput` step that compares compressed versus original length and retains raw output pointers when compression increases token count【/cache/repos/github.com/diegosouzapw/OmniRoute/main/open-sse/services/compression/engines/rtk/index.ts#L44-L48】. Additionally, the final `CompressionStats` record tracks both values, allowing the system to prefer uncompressed content when it yields better efficiency.

### Which specific techniques does the Caveman engine use to reduce tokens?

The Caveman engine applies rules loaded from [`cavemanRules.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/cavemanRules.ts) that remove redundant system-prompt tokens, collapse whitespace and duplicate lines, and enforce language-pack settings【/cache/repos/github.com/diegosouzapw/OmniRoute/main/open-sse/services/compression/engines/cavemanAdapter.ts#L92-L108】. These transformations specifically target the prose and formatting of messages rather than tool-specific output structures handled by RTK.