# How the RTK + Caveman Token Compression Pipeline Works in OmniRoute

> Discover how the RTK + Caveman token compression pipeline works in OmniRoute. Reduce prompt sizes by 30-70% while preserving semantic integrity through a novel two-stage system.

- Repository: [Diego Rodrigues de Sa e Souza/OmniRoute](https://github.com/diegosouzapw/OmniRoute)
- Tags: internals
- Published: 2026-07-23

---

**The RTK + Caveman token compression pipeline is a two-stage system that first applies rule-based semantic condensation via the Caveman engine, then performs token-aware fine-tuning via the RTK engine to reduce prompt sizes by 30-70% while preserving semantic integrity.**

The `diegosouzapw/OmniRoute` repository implements an aggressive request-level compression system designed to minimize upstream LLM costs. This pipeline combines coarse-grained rule matching with specialized content trimming to shrink multi-turn conversations, code diffs, and structured logs before they reach the provider.

## Pipeline Orchestration and Strategy Selection

When a request triggers **aggressive** or **ultra** compression modes, the **strategy selector** ([`open-sse/services/compression/strategySelector.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/strategySelector.ts)) constructs a stacked execution plan. This orchestrator intentionally runs the **Caveman** engine first, then passes the intermediate result to the **RTK** engine ([`open-sse/services/compression/engines/rtk/index.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/engines/rtk/index.ts)). 

This ordering maximizes efficiency: Caveman performs high-level semantic condensation, removing structural redundancy, while RTK applies low-level, content-aware optimizations that require the noise already removed.

## Stage 1: The Caveman Engine

Located in [`open-sse/services/compression/caveman.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/caveman.ts), the Caveman engine executes **rule-based semantic condensation** defined in [`cavemanRules.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/cavemanRules.ts). It operates through three primary mechanisms:

- **Duplicate elimination**: Removes repeated system prompts and redundant messages across multi-turn conversations using deduplication heuristics.
- **Whitespace normalization**: Collapses unnecessary whitespace and formatting characters that consume tokens without adding meaning.
- **Hedged preservation**: Retains a small "hedge" of original content for potential re-injection, ensuring critical information like tool results or code snippets survive the initial pass.

```typescript
// Inside open-sse/services/compression/caveman.ts
export function dedupSystemPrompt(messages: Message[]): Message[] {
  const seen = new Set<string>();
  return messages.filter(m => {
    if (m.role === 'system') {
      if (seen.has(m.content)) return false;
      seen.add(m.content);
    }
    return true;
  });
}

```

## Stage 2: The RTK Engine

The RTK (Rule-based Token-Killer) engine, implemented in [`open-sse/services/compression/engines/rtk/index.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/engines/rtk/index.ts), receives the Caveman-processed content and applies fine-grained, token-specific optimizations through several specialized sub-modules.

### Line Filtering and Cleaning

The [`lineFilter.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/lineFilter.ts) module eliminates blank lines, comment-only lines, and trivial markup that passed through the initial stage. This targets vertical whitespace and non-semantic formatting common in logs and diffs.

### Smart Truncation for Large Blocks

[`smartTruncate.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/smartTruncate.ts) detects the longest meaningful prefix of large content blocks—such as extensive logs or Git diffs—and discards the tail while preserving syntactically valid fragments. It uses the tokenizer to ensure truncation occurs at token boundaries rather than character boundaries.

```typescript
// Inside open-sse/services/compression/engines/rtk/smartTruncate.ts
export function smartTruncate(text: string, maxTokens: number): string {
  const tokens = tokenizer.encode(text);
  if (tokens.length <= maxTokens) return text;
  // keep the first N tokens that preserve a syntactically valid block
  const prefix = tokens.slice(0, maxTokens);
  return tokenizer.decode(prefix);
}

```

### Code-Specific Optimization

[`codeStriper.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/codeStriper.ts) removes language-specific comment syntaxes and redundant boilerplate code. It identifies common patterns like Python docstrings or JavaScript inline comments and strips them without affecting executable logic.

### Cross-Block Deduplication

The [`deduplicator.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/deduplicator.ts) module performs a second-pass deduplication across the entire output, catching repeated lines that may have originated from different sources but converged after the Caveman stage.

### Specialized Renderers

The `renderers/*.ts` collection converts complex structured data into concise textual representations. For example, [`terraformPlan.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/terraformPlan.ts) extracts only resource changes (add, modify, delete), while [`structuredTable.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/structuredTable.ts) preserves header rows with a limited sample of data rows. These renderers transform verbose machine outputs into token-efficient summaries.

## Result Memoization and Metrics

After RTK processing completes, [`resultMemo.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/resultMemo.ts) caches the compressed payload. Subsequent calls within the same session reuse this cached result, eliminating redundant CPU cycles and ensuring consistent token counts across repeated requests.

The [`stats.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/stats.ts) module tracks compression efficacy, logging original token counts, compressed counts, and percentage savings. Unit tests such as [`rtk-smart-truncate.test.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/rtk-smart-truncate.test.ts) and [`caveman-preservation.test.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/caveman-preservation.test.ts) verify that realistic workloads—including tool-generated code diffs and long chat histories—consistently achieve **30-70%** token reduction.

## Implementation Example

To invoke the stacked pipeline in your application:

```typescript
import { resolveCompressionPlan } from '@/open-sse/services/compression/resolveCompressionPlan';
import { CompressionMode } from '@/open-sse/services/compression/types';

const plan = await resolveCompressionPlan({
  mode: CompressionMode.Aggressive,   // triggers Caveman → RTK stacking
  requestBody: incomingPrompt,
});

const compressed = await plan.run();   // runs Caveman then RTK

```

## Summary

- The **RTK + Caveman token compression pipeline** uses a two-stage architecture: Caveman for coarse semantic condensation, RTK for fine-grained token optimization.
- **Caveman** ([`caveman.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/caveman.ts)) removes duplicate system prompts, collapses whitespace, and applies preservation heuristics via rules defined in [`cavemanRules.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/cavemanRules.ts).
- **RTK** ([`engines/rtk/index.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/engines/rtk/index.ts)) performs line filtering, smart truncation, code stripping, and specialized rendering through modular sub-components.
- **Result memoization** ([`resultMemo.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/resultMemo.ts)) caches outputs to improve performance, while **stats** ([`stats.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/stats.ts)) tracks 30-70% token savings.
- The pipeline is triggered via `CompressionMode.Aggressive` or `CompressionMode.Ultra` through the strategy selector ([`strategySelector.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/strategySelector.ts)).

## Frequently Asked Questions

### What is the difference between the Caveman and RTK engines in the compression pipeline?

**Caveman** performs high-level, rule-based semantic condensation such as deduplicating system prompts and normalizing whitespace, while **RTK** executes fine-grained, content-aware optimizations like token-level truncation and code-comment stripping. The pipeline intentionally runs Caveman first to remove structural noise before RTK applies its computationally heavier heuristics.

### How much token reduction can I expect when using the RTK + Caveman pipeline?

According to the test suites [`rtk-smart-truncate.test.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/rtk-smart-truncate.test.ts) and [`caveman-preservation.test.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/caveman-preservation.test.ts), the pipeline consistently achieves **30-70%** token reduction for realistic workloads including multi-turn chat histories, tool-generated code diffs, and verbose structured logs.

### Does the compression pipeline preserve code syntax and semantic meaning?

Yes. The Caveman engine uses **preservation heuristics** to retain critical content like code snippets and tool results, while the RTK engine's [`smartTruncate.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/smartTruncate.ts) ensures truncation occurs at syntactically valid boundaries. The [`codeStriper.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/codeStriper.ts) module specifically targets comment syntax rather than executable code, maintaining semantic integrity.

### How does OmniRoute optimize performance when compressing multiple requests?

The [`resultMemo.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/resultMemo.ts) module implements **result memoization**, storing the compressed output after the first RTK pass. Subsequent requests within the same session retrieve the cached payload instead of reprocessing the original content, significantly reducing CPU overhead and ensuring consistent token counts.