# How Prompt Compression Works in OmniRoute: RTK and Caveman Codecs Explained

> Discover how OmniRoute's prompt compression uses RTK and Caveman codecs to slash token costs with semantic filtering and aggressive heuristics via stacked API request pre-processing.

- Repository: [Diego Rodrigues de Sa e Souza/OmniRoute](https://github.com/diegosouzapw/OmniRoute)
- Tags: deep-dive
- Published: 2026-07-25

---

**Prompt compression in OmniRoute reduces token costs by pre-processing API requests through a stacked pipeline where the RTK engine applies semantic, command-aware filters first, followed by the Caveman engine's aggressive rule-based heuristics.**

OmniRoute implements a modular compression layer that intercepts outgoing LLM requests to minimize token usage before transmission to providers. The system supports multiple compression engines that can be stacked in sequence, with RTK (Rule-Based Tool-output Killer) and Caveman serving as the two primary codecs for shrinking tool outputs and code blocks. Both engines operate on a normalized message format and report detailed statistics through the `createCompressionStats()` utility.

## The Compression Pipeline Architecture

The compression workflow follows a strict pipeline defined in [`open-sse/services/compression/strategySelector.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/strategySelector.ts). This architecture ensures that transformations occur in a predictable order with full observability.

### Message Adaptation

The pipeline begins with `adaptBodyForCompression()` located in [`open-sse/services/compression/bodyAdapter.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/bodyAdapter.ts). This function normalizes the OpenAI-compatible request shape into a uniform `Message[]` array, allowing downstream engines to operate on a consistent data structure regardless of the specific provider format.

### Engine Selection and Stacking

The [`strategySelector.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/strategySelector.ts) module inspects the user's compression settings—specifically `engines.rtk.enabled` and `engines.caveman.enabled`—to construct an execution pipeline. When both engines are active, the system stacks them according to their declared `stackPriority` values:

- **RTK**: `stackPriority = 10`
- **Caveman**: `stackPriority = 20`

Because the selector arranges engines in ascending priority order, the execution flow becomes **RTK → Caveman**. Each engine's `apply()` method receives the request body and returns a modified body along with per-engine statistics, which `createCompressionStats()` in [`open-sse/services/compression/stats.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/stats.ts) aggregates into a final report.

### Execution Flow

```typescript
Raw request 
  → adaptBodyForCompression() 
  → [RTK Engine] 
  → [Caveman Engine] 
  → restoreBody() 
  → API response with compression stats

```

## RTK Engine: Semantic Tool-Output Compression

The RTK engine, implemented in [`open-sse/services/compression/engines/rtk/index.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/engines/rtk/index.ts), targets `tool_results` and `code_blocks` using command-aware, declarative filters. It emphasizes semantic understanding over brute-force truncation.

### Configuration and Eligibility

The engine initializes via `mergeRtkConfig()` (lines 77-104), which combines default settings with user-provided configurations for fields like `intensity`, `maxLinesPerResult`, and filter lists. The `shouldCompressMessage()` function (lines 27-43) determines eligibility based on `applyTo…` flags, ensuring the engine only processes tool results, code blocks, or assistant messages as configured.

### Command Detection and Filter Matching

RTK identifies content type through `detectCommandType()` in [`commandDetector.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/commandDetector.ts), classifying inputs as `docker-logs`, `git-diff`, `generic-output`, or other command types. The engine then invokes `matchRtkFilter()` from [`filterLoader.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/filterLoader.ts) to select a declarative filter whose `id` matches the detected command type and intensity level. Disabled filters are automatically skipped.

### Compression Techniques

The RTK engine applies several transformation techniques in sequence:

- **`applyLineFilter()`**: Drops or trims lines according to the filter's budget (`maxLines`) and priority patterns, preserving high-value content like error lines.
- **Semantic Renderers**: When `enableRenderers` is true, `applyRenderer()` executes specialized renderers (e.g., `terraformPlan`, `gitDiff`) that rewrite verbose outputs into concise summaries.
- **`processRtkText()`**: Removes code-fence sections and optionally strips comments via `stripCodeComments` while preserving doc-strings (lines 76-92).
- **`deduplicateRepeatedLines()`**: Collapses repeated lines to eliminate redundancy.
- **`groupSimilarLines()`**: Merges lines with high similarity (opt-in via configuration).
- **`smartTruncate()`**: Enforces line/character budgets scaled by the `intensity` setting (`minimal`, `standard`, `aggressive`), protecting stack traces and error patterns.

### Raw Output Retention

If compression yields a smaller payload, `maybePersistRtkRawOutput()` (lines 44-53) stores the original text for debugging, creating a pointer recorded in the statistics under `techniquesUsed` as `rtk-raw-output-retention`.

## Caveman Engine: Aggressive Rule-Based Compression

Caveman serves as a fallback and aggressive secondary compressor implemented in [`open-sse/services/compression/engines/cavemanAdapter.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/engines/cavemanAdapter.ts). It executes after RTK with `stackPriority = 20`, operating on hard-coded heuristics rather than declarative filters.

### Rule-Driven Architecture

The engine maintains a static array of rules including `removeNoiseLines`, `collapseLargeBlocks`, and `stripStackTraces`. Each rule specifies regex patterns, actions, and priorities. The `applyCavemanCompression()` function iterates through these rules, applying the first match to each line or block. Rules may trim extremely long lines, drop boilerplate sections (e.g., "Generated by..."), or preserve identifiers matching a protected whitelist.

### Intensity and Safety Controls

Caveman supports three intensity levels—`lite`, `full`, and `ultra`—that scale the maximum allowed lines via `effectiveMaxLines()`. When `autoClarity` is enabled (default: `true`), the engine applies extra safety heuristics in `full` intensity mode to maintain essential readability. Unlike RTK, Caveman disables raw-output retention by default to minimize latency, though it can be enabled via configuration.

### Reported Techniques

The engine reports specific techniques in the `engineBreakdown` statistics:
- `caveman-noise-drop`: Removes lines matching common noise patterns.
- `caveman-line-trim`: Cuts overly long lines to safe lengths.
- `caveman-preserve-identifiers`: Retains symbols from the protected list.

## Engine Stacking and Interaction

When both engines are enabled, the pipeline processes content sequentially:

1. **RTK First**: Applies semantic renderers and command-specific filters, reducing the payload while preserving structure.
2. **Caveman Second**: Receives the already-compressed output and applies aggressive heuristics to remove remaining noise.

This ordering ensures that RTK's semantic understanding (e.g., recognizing a `git diff` and formatting it appropriately) occurs before Caveman's pattern-based truncation. If only one engine is enabled, the pipeline bypasses the other entirely. If both are disabled, requests transmit without compression.

## Configuration and Usage Examples

### Enable Both Engines with Aggressive RTK

```json
{
  "engines": {
    "rtk": {
      "enabled": true,
      "intensity": "aggressive",
      "enableRenderers": true
    },
    "caveman": {
      "enabled": true,
      "intensity": "full"
    }
  }
}

```

This configuration runs semantic renderers and aggressive line filtering in RTK, followed by Caveman's noise removal. The resulting `stats.engineBreakdown` lists techniques like `rtk-render:terraformPlan`, `rtk-truncate`, and `caveman-noise-drop`.

### Caveman-Only Lite Mode

```json
{
  "engines": {
    "caveman": {
      "enabled": true,
      "intensity": "lite"
    }
  }
}

```

Bypasses RTK entirely. Caveman removes obvious boilerplate and caps line lengths, typically achieving 5-10% token savings with negligible latency.

### RTK-Only with Standard Intensity

```json
{
  "engines": {
    "rtk": { 
      "enabled": true, 
      "intensity": "standard" 
    },
    "caveman": { 
      "enabled": false 
    }
  }
}

```

Useful when you need RTK's semantic renderers (e.g., converting verbose `git diff` output into summaries) without Caveman's aggressive truncation.

## Key Source Files

| Purpose | File Path |
|---------|-----------|
| RTK engine core | [`open-sse/services/compression/engines/rtk/index.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/engines/rtk/index.ts) |
| RTK configuration schema | [`open-sse/services/compression/engines/rtk/configSchema.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/engines/rtk/configSchema.ts) |
| Caveman engine | [`open-sse/services/compression/engines/cavemanAdapter.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/engines/cavemanAdapter.ts) |
| Pipeline construction | [`open-sse/services/compression/strategySelector.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/strategySelector.ts) |
| Statistics aggregation | [`open-sse/services/compression/stats.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/stats.ts) |
| Request normalization | [`open-sse/services/compression/bodyAdapter.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/bodyAdapter.ts) |

## Summary

- **Prompt compression in OmniRoute** occurs before provider transmission through a modular pipeline defined in [`strategySelector.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/strategySelector.ts).
- **RTK (priority 10)** provides semantic, command-aware compression using declarative filters and optional renderers for specific output types like `git diff` or `terraform plan`.
- **Caveman (priority 20)** applies aggressive, rule-based heuristics after RTK to remove noise and enforce line budgets.
- The **stacking order** ensures semantic processing precedes brute-force truncation, maximizing token savings while preserving critical information.
- Both engines report detailed statistics via `createCompressionStats()`, including techniques used and raw-output retention pointers.

## Frequently Asked Questions

### What is the difference between RTK and Caveman compression?

RTK uses declarative filters and semantic renderers to understand command output types (like Docker logs or Git diffs) and compress them intelligently, while Caveman applies aggressive, regex-based rules to remove noise and trim lines without context awareness. RTK runs first to preserve semantic structure, and Caveman follows to maximize token reduction.

### How do I configure both engines to work together?

Set both `engines.rtk.enabled` and `engines.caveman.enabled` to `true` in your configuration JSON. The [`strategySelector.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/strategySelector.ts) module automatically orders them by `stackPriority` (RTK at 10, Caveman at 20), ensuring RTK processes the content first and Caveman receives the intermediate result.

### Can I use Caveman without RTK?

Yes. Set `engines.rtk.enabled` to `false` and `engines.caveman.enabled` to `true`. In this mode, requests bypass all RTK-specific logic including semantic renderers and deduplication, relying solely on Caveman's rule-based heuristics for compression.

### How does OmniRoute handle compression statistics?

The `createCompressionStats()` function in [`open-sse/services/compression/stats.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/stats.ts) aggregates results from each engine in the pipeline, recording original token counts, compressed token counts, specific techniques applied (e.g., `rtk-filter`, `caveman-noise-drop`), and pointers to retained raw outputs for auditing.