# How the RTK + Caveman Compression Pipeline Achieves Token Savings in OmniRoute

> Discover how the RTK + Caveman compression pipeline saves tokens in OmniRoute. Achieve 30–70% reductions via deduplication, smart truncation, and efficient data rendering.

- Repository: [Diego Rodrigues de Sa e Souza/OmniRoute](https://github.com/diegosouzapw/OmniRoute)
- Tags: deep-dive
- Published: 2026-07-13

---

**The RTK + Caveman compression pipeline achieves token savings by stacking a coarse-grained semantic condenser (Caveman) with a fine-grained token-aware trimmer (RTK), yielding 30–70% reductions through deduplication, smart truncation, and specialized rendering of structured data.**

The **OmniRoute** repository implements a sophisticated request-level prompt compression system designed to minimize token consumption when sending requests to upstream LLM providers. The **RTK + Caveman compression pipeline** combines two distinct engines—Caveman for coarse rule-based condensation and RTK (Rule-based Token-Killer) for precision trimming—to preserve semantic meaning while dramatically reducing payload size.

## Pipeline Architecture: Stacking Caveman and RTK

### Strategy Selection and Orchestration

When aggressive or "ultra" compression modes are activated, the **strategy selector** located in [`open-sse/services/compression/strategySelector.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/strategySelector.ts) constructs a stacked execution plan. This plan intentionally runs the **Caveman** engine first, followed by the **RTK** engine in [`open-sse/services/compression/engines/rtk/index.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/engines/rtk/index.ts).

### Execution Order and Rationale

The ordering matters: **Caveman** performs high-level semantic condensation first, removing duplicate system prompts and collapsing whitespace. The partially compressed output then flows into **RTK**, which applies fine-grained, content-aware trimming such as line filtering and smart truncation. This sequential approach ensures that coarse redundancies are eliminated before token-level optimization begins.

## Stage 1: The Caveman Engine

### Rule-Based Semantic Condensation

The **Caveman** engine ([`open-sse/services/compression/caveman.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/caveman.ts)) applies handcrafted rules defined in [`open-sse/services/compression/cavemanRules.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/cavemanRules.ts). These rules remove duplicate system prompts, strip irrelevant sections, and collapse whitespace while applying preservation heuristics to retain critical information like code snippets and tool results.

### Deduplication and Hedging

Caveman deduplicates repeated messages across multi-turn conversations using a Set-based tracking mechanism. The engine also maintains a small "hedge" of original content for potential re-injection, ensuring that aggressive compression does not permanently lose context that might be needed later.

```typescript
// Inside the Caveman engine – deduplicating system prompts
export function dedupSystemPrompt(messages: Message[]): Message[] {
  const seen = new Set<string>();
  return messages.filter(m => {
    if (m.role === 'system') {
      if (seen.has(m.content)) return false;
      seen.add(m.content);
    }
    return true;
  });
}

```

## Stage 2: The RTK Engine

### Line Filtering and Smart Truncation

The **RTK** engine processes Caveman's output through several specialized modules. The **line filter** ([`open-sse/services/compression/engines/rtk/lineFilter.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/engines/rtk/lineFilter.ts)) eliminates blank lines, comment lines, and trivial markup. The **smart truncation** module ([`open-sse/services/compression/engines/rtk/smartTruncate.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/engines/rtk/smartTruncate.ts)) detects the longest meaningful prefix of large blocks—such as lengthy logs or diffs—and discards the tail while preserving syntactically valid fragments.

### Code Stripping and Deduplication

The **code stripper** ([`open-sse/services/compression/engines/rtk/codeStriper.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/engines/rtk/codeStriper.ts)) removes language-specific comment syntaxes and redundant boilerplate. The **deduplicator** ([`open-sse/services/compression/engines/rtk/deduplicator.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/engines/rtk/deduplicator.ts)) further collapses repeated lines across the Caveman stage output, ensuring no redundancy persists into the final payload.

### Specialized Renderers for Structured Data

RTK includes domain-specific renderers in `open-sse/services/compression/engines/rtk/renderers/` that convert complex structures into token-efficient text. For example, [`terraformPlan.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/terraformPlan.ts) extracts only resource changes, while [`structuredTable.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/structuredTable.ts) preserves header rows with minimal sample data, drastically reducing token count for structured outputs.

```typescript
// Inside the RTK engine – smart truncation of long log blocks
export function smartTruncate(text: string, maxTokens: number): string {
  const tokens = tokenizer.encode(text);
  if (tokens.length <= maxTokens) return text;
  // Keep the first N tokens that preserve a syntactically valid block
  const prefix = tokens.slice(0, maxTokens);
  return tokenizer.decode(prefix);
}

```

## Performance Optimization and Caching

### Result Memoization

After RTK completes processing, the **result memo** module ([`open-sse/services/compression/resultMemo.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/resultMemo.ts)) caches the compressed result. Subsequent calls within the same session reuse this payload without re-processing, reducing CPU overhead and maintaining consistent token counts.

### Token Savings Metrics

The **stats** module ([`open-sse/services/compression/stats.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/stats.ts)) records original versus compressed token counts and calculates percentage savings. Unit tests in [`rtk-smart-truncate.test.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/rtk-smart-truncate.test.ts) and [`caveman-preservation.test.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/caveman-preservation.test.ts) verify that the pipeline consistently achieves **30–70%** token reductions for realistic workloads including tool-generated diffs, long logs, and multi-turn chat histories.

```typescript
// Example: Using the stacked "aggressive" mode
import { resolveCompressionPlan } from '@/open-sse/services/compression/resolveCompressionPlan';
import { CompressionMode } from '@/open-sse/services/compression/types';

const plan = await resolveCompressionPlan({
  mode: CompressionMode.Aggressive,   // Triggers Caveman → RTK stacking
  requestBody: incomingPrompt,
});

const compressed = await plan.run();   // Runs Caveman then RTK

```

## Summary

- **Two-stage architecture**: Caveman handles coarse semantic condensation while RTK performs fine-grained token optimization.
- **Strategic ordering**: Running Caveman before RTK eliminates high-level redundancy before line-level trimming occurs.
- **Comprehensive deduplication**: Both engines remove duplicates at different granularities—Caveman at the message level and RTK at the line level.
- **Smart truncation**: RTK preserves syntactically valid prefixes when truncating large content blocks rather than cutting arbitrarily.
- **Specialized renderers**: Domain-specific parsers in `open-sse/services/compression/engines/rtk/renderers/` convert structured data (Terraform plans, tables, diffs) into minimal token representations.
- **Proven savings**: The pipeline consistently achieves **30–70%** token reduction across production workloads, verified by the **stats** module and comprehensive unit tests.

## Frequently Asked Questions

### How does the ordering of Caveman and RTK affect compression quality?

**Caveman runs before RTK** to ensure that high-level semantic redundancies—such as duplicate system prompts and repeated conversation turns—are eliminated first. This prevents RTK from wasting token budget on trimming content that should have been removed entirely, allowing RTK to focus its fine-grained heuristics on preserving the semantic density of the remaining content.

### What types of content benefit most from the RTK + Caveman pipeline?

**Structured data and repetitive conversations** see the highest savings. Terraform plans processed by [`terraformPlan.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/terraformPlan.ts), long log files handled by [`smartTruncate.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/smartTruncate.ts), and multi-turn chat histories with repeated system prompts benefit most—often hitting the upper range of **70%** token reduction according to the **stats** module in [`open-sse/services/compression/stats.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/stats.ts).

### How does result memoization improve performance beyond token savings?

The **result memo** module ([`open-sse/services/compression/resultMemo.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/resultMemo.ts)) caches the final compressed output after RTK processing. This eliminates redundant CPU cycles for identical subsequent requests in the same session, ensuring that compression overhead remains minimal while keeping token counts consistent across repeated calls.

### What token reduction rates can be expected in production workloads?

According to unit tests in [`caveman-preservation.test.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/caveman-preservation.test.ts) and [`rtk-smart-truncate.test.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/rtk-smart-truncate.test.ts), the **RTK + Caveman compression pipeline** consistently delivers **30–70%** token savings. The actual percentage depends on content redundancy—highly repetitive tool outputs (like build logs or infrastructure diffs) typically achieve higher savings than dense, unique prose.