OmniRoute Prompt Compression Pipeline Architecture: RTK and Caveman Modes Explained

OmniRoute implements a dual-engine prompt compression system that combines rule-based semantic cleanup (Caveman) with command-aware tool-result optimization (RTK) to reduce token usage before upstream LLM requests.

The diegosouzapw/OmniRoute repository provides a sophisticated compression framework that intercepts request bodies sent to LLM providers and applies configurable transformations to minimize token counts. This architecture supports both standalone engine execution and stacked pipelines where multiple compression strategies run sequentially. Understanding these mechanisms enables developers to optimize costs while preserving semantic integrity across natural language and structured tool outputs.

How the Pipeline Is Chosen

The entry point for all compression operations is applyCompression in strategySelector.ts. This function serves as the orchestration layer that determines which compression strategy to execute based on runtime configuration and request characteristics.

The selection process follows three deterministic steps:

1. Mode Resolution: The getEffectiveMode function evaluates global configuration (CompressionConfig), combo overrides (comboId), estimated token counts, and named profiles to return a specific compression mode (off, lite, standard, aggressive, ultra, rtk, stacked, etc.).

2. Pipeline Construction: For stacked modes, resolveStackSteps builds an ordered array of CompressionPipelineStep objects. When no explicit pipeline is defined, the system defaults to ["rtk", "caveman"] execution order.

3. Engine Dispatch: The selector invokes the appropriate engine runner—either applyRtkCompression, cavemanCompress, or runStackedCompression for multi-engine sequences.

// Source: strategySelector.ts
export function getEffectiveMode(
  config,
  comboId,
  estimatedTokens,
  combos = {},
  header = null
): CompressionMode {
  return resolveBasePlan(config, comboId, estimatedTokens, combos, header).mode;
}

Caveman Engine: Rule-Based Semantic Cleanup

The Caveman engine handles semantic cleanup of natural language content through regex-based transformations defined in caveman.ts. This engine specializes in removing conversational filler, normalizing whitespace, and applying language-specific compression rules while protecting structural elements like code fences and URLs.

Core Compression Flow

The cavemanCompress function processes each message in the request body through a deterministic pipeline:

1. Text Extraction: Pulls plain text from string content or array-based content blocks
2. Structure Detection: Uses hasProtectedStructure with PROTECTED_STRUCTURE_RE to identify code blocks, markdown, URLs, and error traces that must remain unaltered
3. Rule Application: Selects language-specific rule packs via getRulesForContext and applies regex transformations through applyRulesToText
4. Artifact Cleanup: Collapses whitespace, trims punctuation, normalizes newlines, and recapitalizes sentences via cleanupArtifacts and recapitalizeSentences
5. Validation: Re-inserts protected blocks and verifies structural integrity
6. Telemetry: Generates statistics via createCavemanStats tracking original vs. compressed token counts and applied techniques

// Source: caveman.ts
const { text: rulesApplied, appliedRules } = applyRulesToText(extractedText, rules);
const normalized = recapitalizeSentences(cleanupArtifacts(rulesApplied));
const stats = createCavemanStats(originalTokens, compressedTokens, uniqueRules, …);

Protected Structures and Language-Aware Rules

Caveman preserves critical technical content through protected structure detection. The hasProtectedStructure function employs a comprehensive regex pattern to identify code fences, error traces, and URLs that would break if compressed.

Language detection (detectCompressionLanguage) enables contextual rule application from defined language packs in cavemanRules.ts. The engine filters rule candidates using keyword heuristics (shouldAttemptRule) and the RULE_KEYWORDS table to avoid false positives on polite filler phrases like "please" and "thank you".

RTK Engine: Command-Aware Tool-Result Compression

The RTK engine (Real-Time Kompresion) in engines/rtk/index.ts handles compression of tool outputs and command results. Unlike Caveman's general-purpose text cleanup, RTK understands command contexts, enabling intelligent truncation of shell output, structured data formatting, and deduplication of repetitive content.

Tool-Call Lookup and Message Processing

RTK maintains a toolCallLookup Map that correlates tool results with their originating tool calls from preceding assistant messages. This mapping supports both OpenAI (tool_calls) and Anthropic (tool_use) formats.

The shouldCompressMessage function determines eligibility for compression based on message role (tool, tool_result, or assistant with code blocks). For Anthropic's tool_result blocks, RTK compresses inner text content while preserving outer block metadata including cache_control markers.

// Source: engines/rtk/index.ts
const toolCallLookup = new Map<string, ToolMeta>();
// parse assistant messages → toolCallLookup

Transformation Pipeline

The processRtkContent function applies a sophisticated multi-stage transformation pipeline:

1. Command Detection: detectCommandType identifies the originating shell command to enable context-aware filtering
2. Filter Application: matchRtkFilter and applyLineFilter apply declarative line and character truncation rules
3. Semantic Rendering: applyRenderer optionally transforms known output types (e.g., pretty-printing git diff output)
4. Code Processing: stripCode removes comments and optionally preserves docstrings within code blocks
5. Deduplication: deduplicateRepeatedLines removes redundant output lines
6. Grouping: groupSimilarLines consolidates similar output patterns
7. Smart Truncation: smartTruncate implements head/tail preservation with priority pattern matching

// Source: engines/rtk/index.ts
const stats = createCompressionStats(...);

Stacked Pipeline Architecture

OmniRoute supports stacked pipelines that chain multiple engines sequentially. When the mode resolves to stacked or when an explicit stackedPipeline configuration is provided, the runStackedCompression function executes engines in defined order.

The stack loop performs the following operations:

1. Resolves each pipeline step via resolveStackSteps
2. Validates engine availability against the registry (getCompressionEngine)
3. Executes each engine synchronously or asynchronously
4. Records per-step telemetry via reportEngineStep
5. Applies hard-budget guards through applyHardBudget after all transformations

This architecture enables workflows like rtk → caveman → hard-budget, where command-aware compression precedes semantic cleanup, followed by absolute token limits.

Configuration and Performance Optimization

The engines/rtk/configSchema.ts file defines the RtkConfig interface governing RTK behavior:

Configuration	Effect
intensity	Scales line budgets across minimal, standard, and aggressive levels
enabledFilters / disabledFilters	Whitelist or blacklist specific filter IDs
applyToToolResults	Enables compression of OpenAI and Anthropic tool-result blocks
applyToCodeBlocks	Activates code-fence processing for assistant messages
enableRenderers	Turns on semantic rendering for known command outputs
customFiltersEnabled	Activates user-defined filter packs

Caveman configuration supports intensity levels and language-specific rule enabling through cavemanConfig in the global compression configuration.

Request Flow and Integration

The compression pipeline integrates into OmniRoute's request lifecycle through chatCore.ts, which receives incoming API requests and delegates to selectCompressionPlan → selectCompressionStrategy → applyCompression.

After compression, the system forwards the modified body to the appropriate provider executor. The entire flow maintains detailed telemetry through stats.ts, enabling observability of token savings and applied techniques.

Standard Mode (Caveman) Example

import { applyCompression } from '@omniroute/open-sse/services/compression';

const body = {
  messages: [
    { role: 'user', content: 'Please be sure to answer the question. Thanks!' },
  ],
};

const result = applyCompression(body, 'standard', {
  config: {
    cavemanConfig: { intensity: 'full' },
    preserveSystemPrompt: true,
  },
});

console.log(result.compressed); // true if tokens were saved
console.log(result.stats?.techniquesUsed); // e.g. ["caveman-rules"]

RTK Tool-Result Compression Example

import { applyCompression } from '@omniroute/open-sse/services/compression';

const body = {
  messages: [
    {
      role: 'tool',
      tool_call_id: 'call-123',
      content: `\`\`\`bash\nError: something failed\nLine 1\nLine 2\n\`\`\``,
    },
  ],
};

const result = applyCompression(body, 'rtk', {
  config: {
    rtkConfig: {
      enabled: true,
      applyToToolResults: true,
      intensity: 'aggressive',
      enabledFilters: ['bash-errors'],
    },
  },
});

console.log(result.stats?.techniquesUsed); // e.g. ["rtk-filter","rtk-truncate"]

Custom Stacked Pipeline Example

import { applyCompression } from '@omniroute/open-sse/services/compression';

const stackedPipeline = [
  { engine: 'rtk', intensity: 'standard' },
  { engine: 'caveman', intensity: 'full' },
];

const result = applyCompression(body, 'stacked', {
  config: {
    stackedPipeline,
    rtkConfig: { enabled: true, applyToToolResults: true },
    cavemanConfig: { enabled: true },
  },
});

Summary

• Caveman provides rule-based semantic cleanup using regex transformations, protected structure detection, and language-aware rule packs to compress natural language while preserving code blocks and URLs.
• RTK delivers command-aware tool-result compression with configurable filters, deduplication, grouping, and smart truncation specifically optimized for shell output and structured tool results.
• The strategy selector orchestrates engine execution through applyCompression, supporting both single-mode operation and stacked pipelines with detailed telemetry collection.
• Configuration schemas in configSchema.ts enable fine-grained control over intensity levels, filter activation, and budget enforcement via hardBudget.ts.

Frequently Asked Questions

What is the difference between Caveman and RTK compression modes?

Caveman applies general-purpose semantic cleanup to natural language text, removing filler words, normalizing whitespace, and applying language-specific abbreviation rules. RTK specifically targets tool outputs and command results, understanding shell contexts to perform intelligent truncation, deduplication, and grouping. While Caveman processes all message types, RTK focuses on tool roles and tool_result blocks from both OpenAI and Anthropic APIs.

How does OmniRoute protect code blocks and URLs during compression?

The Caveman engine uses hasProtectedStructure with the PROTECTED_STRUCTURE_RE regex pattern to identify markdown code fences, URLs, error traces, and other technical content. These structures are extracted before rule application and re-inserted afterward, ensuring that regex-based transformations never corrupt syntactically sensitive content. This protection mechanism operates in caveman.ts during the cavemanCompress execution cycle.

Can I combine multiple compression engines in a single request?

Yes. OmniRoute supports stacked pipelines through the stacked mode in strategySelector.ts. You can define an ordered array of engines—such as ["rtk", "caveman"]—that execute sequentially. The runStackedCompression function processes each step, maintaining telemetry for every transformation stage and applying hard-budget guards after completion. This enables sophisticated workflows where command-aware truncation precedes semantic cleanup.

What configuration options control RTK compression intensity?

The RtkConfig interface in engines/rtk/configSchema.ts provides granular control through the intensity parameter (minimal, standard, or aggressive), which scales internal line budgets. Additional options include enabledFilters for whitelist control, applyToToolResults for tool-content targeting, enableRenderers for semantic output formatting, and customFiltersEnabled for user-defined rule packs. These settings are validated by validateRtkEngineConfig before pipeline execution.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}