# OmniRoute Prompt Compression Pipeline Architecture: RTK and Caveman Modes Explained

> Explore the OmniRoute prompt compression pipeline architecture. Learn how RTK and Caveman modes optimize token usage for efficient LLM requests.

- Repository: [Diego Rodrigues de Sa e Souza/OmniRoute](https://github.com/diegosouzapw/OmniRoute)
- Tags: architecture
- Published: 2026-07-17

---

**OmniRoute implements a dual-engine prompt compression system that combines rule-based semantic cleanup (Caveman) with command-aware tool-result optimization (RTK) to reduce token usage before upstream LLM requests.**

The **diegosouzapw/OmniRoute** repository provides a sophisticated compression framework that intercepts request bodies sent to LLM providers and applies configurable transformations to minimize token counts. This architecture supports both standalone engine execution and stacked pipelines where multiple compression strategies run sequentially. Understanding these mechanisms enables developers to optimize costs while preserving semantic integrity across natural language and structured tool outputs.

## How the Pipeline Is Chosen

The entry point for all compression operations is `applyCompression` in **[`strategySelector.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/strategySelector.ts)**. This function serves as the orchestration layer that determines which compression strategy to execute based on runtime configuration and request characteristics.

The selection process follows three deterministic steps:

1. **Mode Resolution**: The `getEffectiveMode` function evaluates global configuration (`CompressionConfig`), combo overrides (`comboId`), estimated token counts, and named profiles to return a specific compression mode (`off`, `lite`, `standard`, `aggressive`, `ultra`, `rtk`, `stacked`, etc.).

2. **Pipeline Construction**: For stacked modes, `resolveStackSteps` builds an ordered array of `CompressionPipelineStep` objects. When no explicit pipeline is defined, the system defaults to `["rtk", "caveman"]` execution order.

3. **Engine Dispatch**: The selector invokes the appropriate engine runner—either `applyRtkCompression`, `cavemanCompress`, or `runStackedCompression` for multi-engine sequences.

```ts
// Source: strategySelector.ts
export function getEffectiveMode(
  config,
  comboId,
  estimatedTokens,
  combos = {},
  header = null
): CompressionMode {
  return resolveBasePlan(config, comboId, estimatedTokens, combos, header).mode;
}

```

## Caveman Engine: Rule-Based Semantic Cleanup

The **Caveman** engine handles semantic cleanup of natural language content through regex-based transformations defined in **[`caveman.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/caveman.ts)**. This engine specializes in removing conversational filler, normalizing whitespace, and applying language-specific compression rules while protecting structural elements like code fences and URLs.

### Core Compression Flow

The `cavemanCompress` function processes each message in the request body through a deterministic pipeline:

1. **Text Extraction**: Pulls plain text from string content or array-based content blocks
2. **Structure Detection**: Uses `hasProtectedStructure` with `PROTECTED_STRUCTURE_RE` to identify code blocks, markdown, URLs, and error traces that must remain unaltered
3. **Rule Application**: Selects language-specific rule packs via `getRulesForContext` and applies regex transformations through `applyRulesToText`
4. **Artifact Cleanup**: Collapses whitespace, trims punctuation, normalizes newlines, and recapitalizes sentences via `cleanupArtifacts` and `recapitalizeSentences`
5. **Validation**: Re-inserts protected blocks and verifies structural integrity
6. **Telemetry**: Generates statistics via `createCavemanStats` tracking original vs. compressed token counts and applied techniques

```ts
// Source: caveman.ts
const { text: rulesApplied, appliedRules } = applyRulesToText(extractedText, rules);
const normalized = recapitalizeSentences(cleanupArtifacts(rulesApplied));
const stats = createCavemanStats(originalTokens, compressedTokens, uniqueRules, …);

```

### Protected Structures and Language-Aware Rules

Caveman preserves critical technical content through **protected structure detection**. The `hasProtectedStructure` function employs a comprehensive regex pattern to identify code fences, error traces, and URLs that would break if compressed.

Language detection (`detectCompressionLanguage`) enables contextual rule application from defined language packs in [`cavemanRules.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/cavemanRules.ts). The engine filters rule candidates using keyword heuristics (`shouldAttemptRule`) and the `RULE_KEYWORDS` table to avoid false positives on polite filler phrases like "please" and "thank you".

## RTK Engine: Command-Aware Tool-Result Compression

The **RTK** engine (Real-Time Kompresion) in **[`engines/rtk/index.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/engines/rtk/index.ts)** handles compression of tool outputs and command results. Unlike Caveman's general-purpose text cleanup, RTK understands command contexts, enabling intelligent truncation of shell output, structured data formatting, and deduplication of repetitive content.

### Tool-Call Lookup and Message Processing

RTK maintains a `toolCallLookup` Map that correlates tool results with their originating tool calls from preceding assistant messages. This mapping supports both OpenAI (`tool_calls`) and Anthropic (`tool_use`) formats.

The `shouldCompressMessage` function determines eligibility for compression based on message role (`tool`, `tool_result`, or `assistant` with code blocks). For Anthropic's `tool_result` blocks, RTK compresses inner text content while preserving outer block metadata including `cache_control` markers.

```ts
// Source: engines/rtk/index.ts
const toolCallLookup = new Map<string, ToolMeta>();
// parse assistant messages → toolCallLookup

```

### Transformation Pipeline

The `processRtkContent` function applies a sophisticated multi-stage transformation pipeline:

1. **Command Detection**: `detectCommandType` identifies the originating shell command to enable context-aware filtering
2. **Filter Application**: `matchRtkFilter` and `applyLineFilter` apply declarative line and character truncation rules
3. **Semantic Rendering**: `applyRenderer` optionally transforms known output types (e.g., pretty-printing `git diff` output)
4. **Code Processing**: `stripCode` removes comments and optionally preserves docstrings within code blocks
5. **Deduplication**: `deduplicateRepeatedLines` removes redundant output lines
6. **Grouping**: `groupSimilarLines` consolidates similar output patterns
7. **Smart Truncation**: `smartTruncate` implements head/tail preservation with priority pattern matching

```ts
// Source: engines/rtk/index.ts
const stats = createCompressionStats(...);

```

## Stacked Pipeline Architecture

OmniRoute supports **stacked pipelines** that chain multiple engines sequentially. When the mode resolves to `stacked` or when an explicit `stackedPipeline` configuration is provided, the `runStackedCompression` function executes engines in defined order.

The stack loop performs the following operations:

1. Resolves each pipeline step via `resolveStackSteps`
2. Validates engine availability against the registry (`getCompressionEngine`)
3. Executes each engine synchronously or asynchronously
4. Records per-step telemetry via `reportEngineStep`
5. Applies hard-budget guards through `applyHardBudget` after all transformations

This architecture enables workflows like `rtk → caveman → hard-budget`, where command-aware compression precedes semantic cleanup, followed by absolute token limits.

## Configuration and Performance Optimization

The **[`engines/rtk/configSchema.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/engines/rtk/configSchema.ts)** file defines the `RtkConfig` interface governing RTK behavior:

| Configuration | Effect |
|---------------|--------|
| `intensity` | Scales line budgets across `minimal`, `standard`, and `aggressive` levels |
| `enabledFilters` / `disabledFilters` | Whitelist or blacklist specific filter IDs |
| `applyToToolResults` | Enables compression of OpenAI and Anthropic tool-result blocks |
| `applyToCodeBlocks` | Activates code-fence processing for assistant messages |
| `enableRenderers` | Turns on semantic rendering for known command outputs |
| `customFiltersEnabled` | Activates user-defined filter packs |

Caveman configuration supports intensity levels and language-specific rule enabling through `cavemanConfig` in the global compression configuration.

## Request Flow and Integration

The compression pipeline integrates into OmniRoute's request lifecycle through **[`chatCore.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/chatCore.ts)**, which receives incoming API requests and delegates to `selectCompressionPlan` → `selectCompressionStrategy` → `applyCompression`.

After compression, the system forwards the modified body to the appropriate provider executor. The entire flow maintains detailed telemetry through **[`stats.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/stats.ts)**, enabling observability of token savings and applied techniques.

### Standard Mode (Caveman) Example

```ts
import { applyCompression } from '@omniroute/open-sse/services/compression';

const body = {
  messages: [
    { role: 'user', content: 'Please be sure to answer the question. Thanks!' },
  ],
};

const result = applyCompression(body, 'standard', {
  config: {
    cavemanConfig: { intensity: 'full' },
    preserveSystemPrompt: true,
  },
});

console.log(result.compressed); // true if tokens were saved
console.log(result.stats?.techniquesUsed); // e.g. ["caveman-rules"]

```

### RTK Tool-Result Compression Example

```ts
import { applyCompression } from '@omniroute/open-sse/services/compression';

const body = {
  messages: [
    {
      role: 'tool',
      tool_call_id: 'call-123',
      content: `\`\`\`bash\nError: something failed\nLine 1\nLine 2\n\`\`\``,
    },
  ],
};

const result = applyCompression(body, 'rtk', {
  config: {
    rtkConfig: {
      enabled: true,
      applyToToolResults: true,
      intensity: 'aggressive',
      enabledFilters: ['bash-errors'],
    },
  },
});

console.log(result.stats?.techniquesUsed); // e.g. ["rtk-filter","rtk-truncate"]

```

### Custom Stacked Pipeline Example

```ts
import { applyCompression } from '@omniroute/open-sse/services/compression';

const stackedPipeline = [
  { engine: 'rtk', intensity: 'standard' },
  { engine: 'caveman', intensity: 'full' },
];

const result = applyCompression(body, 'stacked', {
  config: {
    stackedPipeline,
    rtkConfig: { enabled: true, applyToToolResults: true },
    cavemanConfig: { enabled: true },
  },
});

```

## Summary

- **Caveman** provides **rule-based semantic cleanup** using regex transformations, protected structure detection, and language-aware rule packs to compress natural language while preserving code blocks and URLs.
- **RTK** delivers **command-aware tool-result compression** with configurable filters, deduplication, grouping, and smart truncation specifically optimized for shell output and structured tool results.
- The **strategy selector** orchestrates engine execution through `applyCompression`, supporting both single-mode operation and stacked pipelines with detailed telemetry collection.
- **Configuration schemas** in [`configSchema.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/configSchema.ts) enable fine-grained control over intensity levels, filter activation, and budget enforcement via [`hardBudget.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/hardBudget.ts).

## Frequently Asked Questions

### What is the difference between Caveman and RTK compression modes?

**Caveman** applies general-purpose semantic cleanup to natural language text, removing filler words, normalizing whitespace, and applying language-specific abbreviation rules. **RTK** specifically targets tool outputs and command results, understanding shell contexts to perform intelligent truncation, deduplication, and grouping. While Caveman processes all message types, RTK focuses on `tool` roles and `tool_result` blocks from both OpenAI and Anthropic APIs.

### How does OmniRoute protect code blocks and URLs during compression?

The Caveman engine uses `hasProtectedStructure` with the `PROTECTED_STRUCTURE_RE` regex pattern to identify markdown code fences, URLs, error traces, and other technical content. These structures are extracted before rule application and re-inserted afterward, ensuring that regex-based transformations never corrupt syntactically sensitive content. This protection mechanism operates in [`caveman.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/caveman.ts) during the `cavemanCompress` execution cycle.

### Can I combine multiple compression engines in a single request?

Yes. OmniRoute supports **stacked pipelines** through the `stacked` mode in [`strategySelector.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/strategySelector.ts). You can define an ordered array of engines—such as `["rtk", "caveman"]`—that execute sequentially. The `runStackedCompression` function processes each step, maintaining telemetry for every transformation stage and applying hard-budget guards after completion. This enables sophisticated workflows where command-aware truncation precedes semantic cleanup.

### What configuration options control RTK compression intensity?

The `RtkConfig` interface in [`engines/rtk/configSchema.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/engines/rtk/configSchema.ts) provides granular control through the `intensity` parameter (`minimal`, `standard`, or `aggressive`), which scales internal line budgets. Additional options include `enabledFilters` for whitelist control, `applyToToolResults` for tool-content targeting, `enableRenderers` for semantic output formatting, and `customFiltersEnabled` for user-defined rule packs. These settings are validated by `validateRtkEngineConfig` before pipeline execution.