How OmniRoute's Prompt Compression Pipeline Combines RTK and Caveman Modes

OmniRoute's prompt compression pipeline reduces token usage by orchestrating two specialized engines: Caveman for rule-based semantic cleanup of natural language, and RTK for command-aware compression of tool outputs, which can be stacked sequentially via the strategy selector in strategySelector.ts.

OmniRoute implements a sophisticated prompt compression pipeline to minimize token costs before forwarding requests to upstream LLM providers. The system combines the Caveman engine for rule-based text normalization with the RTK engine for intelligent tool-result compression, orchestrated through a configurable strategy selector. This dual-engine architecture allows precise adaptation to both conversational content and structured command outputs.

How the Strategy Selector Chooses the Compression Mode

The entry point for all compression operations is applyCompression in strategySelector.ts, which determines the effective compression mode and dispatches the appropriate engine. The selector evaluates multiple inputs to resolve the mode via getEffectiveMode at lines 79–89:

export function getEffectiveMode(
  config,
  comboId,
  estimatedTokens,
  combos = {},
  header = null
): CompressionMode {
  return resolveBasePlan(config, comboId, estimatedTokens, combos, header).mode;
}

Available modes include off, lite, standard, aggressive, ultra, rtk, and stacked. When the mode is stacked or when an explicit per-engine map exists, the selector builds a processing pipeline using resolveStackSteps at lines 94–110. The default fallback stack is [rtk, caveman], ensuring RTK processes tool outputs first, followed by Caveman's semantic cleanup.

Caveman Engine: Rule-Based Semantic Cleanup

The Caveman engine, implemented in caveman.ts, performs rule-based "semantic cleanup" on natural language content. It operates through the cavemanCompress function, which processes each message in body.messages through a multi-stage pipeline.

First, the engine extracts plain text from strings or array-based content blocks, then detects protected structures—such as code fences, markdown, URLs, and error traces—using hasProtectedStructure and the PROTECTED_STRUCTURE_RE regex to avoid corruption. It selects language-specific rule packs via getRulesForContext based on message role and detected language, then applies regex-based transformations through applyRulesToText at lines 75–84:

const { text: rulesApplied, appliedRules } = applyRulesToText(extractedText, rules);

After applying rules, Caveman cleans up artifacts through whitespace collapse, punctuation trimming, and recapitalization at lines 90–98:

const normalized = recapitalizeSentences(cleanupArtifacts(rulesApplied));

The engine validates that protected structures remain intact, then collects statistics via createCavemanStats at lines 95–104, reporting original and compressed token counts along with applied techniques.

RTK Engine: Command-Aware Tool-Output Compression

The RTK engine, located in engines/rtk/index.ts, specializes in compressing tool-generated outputs through command-aware processing. The applyRtkCompression function adapts the request body, builds a tool-call lookup map from previous assistant messages (supporting both OpenAI tool_calls and Anthropic tool_use), and iterates through messages to determine compressibility via shouldCompressMessage at lines 27–43.

For each compressible message, processRtkContent executes a sophisticated transformation pipeline:

• Command detection: Identifies the originating command via detectCommandType
• Filter application: Applies declarative line filters through matchRtkFilter and applyLineFilter
• Semantic rendering: Optionally pretty-prints known output types (e.g., git diff) via applyRenderer
• Code processing: Strips comments while optionally preserving docstrings using stripCode
• Deduplication: Removes repeated lines via deduplicateRepeatedLines
• Grouping: Consolidates similar output lines through groupSimilarLines
• Smart truncation: Applies head/tail preservation with priority patterns via smartTruncate

For Anthropic-style tool_result blocks, RTK compresses inner text while preserving outer block structure and cache_control markers byte-for-byte at lines 72–87. The engine validates configuration against engines/rtk/configSchema.ts and collects telemetry through createCompressionStats at lines 38–46.

Stacked Pipeline Execution

When stacked mode is selected, the strategy selector orchestrates both engines sequentially through runStackedCompression at lines 310–380 in strategySelector.ts. The pipeline resolver normalizes each step via resolveStackSteps, checks the engine registry, executes compressions, and records per-step telemetry.

A typical stacked configuration processes tool results through RTK first to remove technical noise, then passes the output through Caveman to eliminate conversational filler. The pipeline concludes with a hard-budget guard from hardBudget.ts to enforce absolute token limits.

Implementation Examples

Compressing with Standard (Caveman) Mode

import { applyCompression } from '@omniroute/open-sse/services/compression';

const body = {
  messages: [
    { role: 'user', content: 'Please be sure to answer the question. Thanks!' },
  ],
};

const result = applyCompression(body, 'standard', {
  config: {
    cavemanConfig: { intensity: 'full' },
    preserveSystemPrompt: true,
  },
});

console.log(result.compressed); // true if tokens were saved
console.log(result.stats?.techniquesUsed); // e.g. ["caveman-rules"]

Compressing Tool Results with RTK

import { applyCompression } from '@omniroute/open-sse/services/compression';

const body = {
  messages: [
    {
      role: 'tool',
      tool_call_id: 'call-123',
      content: `\`\`\`bash\nError: something failed\nLine 1\nLine 2\n\`\`\``,
    },
  ],
};

const result = applyCompression(body, 'rtk', {
  config: {
    rtkConfig: {
      enabled: true,
      applyToToolResults: true,
      intensity: 'aggressive',
      enabledFilters: ['bash-errors'],
    },
  },
});

console.log(result.stats?.techniquesUsed); // e.g. ["rtk-filter","rtk-truncate"]

Building a Custom Stacked Pipeline

import { applyCompression } from '@omniroute/open-sse/services/compression';

const stackedPipeline = [
  { engine: 'rtk', intensity: 'standard' },
  { engine: 'caveman', intensity: 'full' },
];

const result = applyCompression(body, 'stacked', {
  config: {
    stackedPipeline,
    rtkConfig: { enabled: true, applyToToolResults: true },
    cavemanConfig: { enabled: true },
  },
});

console.log(result.stats?.techniquesUsed); // includes both RTK and Caveman techniques

Summary

• The strategy selector in strategySelector.ts determines compression mode and orchestrates engine execution, defaulting to a [rtk, caveman] stack when appropriate.
• Caveman applies language-aware regex rules to natural language, removing filler words and normalizing whitespace while protecting code blocks and URLs via PROTECTED_STRUCTURE_RE.
• RTK provides command-aware compression for tool outputs, implementing filters, deduplication, grouping, and smart truncation specifically for structured technical content.
• Both engines support configurable intensity levels and can be combined in stacked pipelines for sequential processing of complex requests.
• Comprehensive telemetry through createCompressionStats and createCavemanStats enables observability of token savings and applied techniques.

Frequently Asked Questions

What is the primary difference between Caveman and RTK compression in OmniRoute?

Caveman focuses on natural language optimization through regex-based rules that remove politeness filler and collapse whitespace, while RTK specializes in command-aware processing of tool outputs with features like deduplication, line grouping, and semantic rendering. Caveman operates universally on message content, whereas RTK specifically targets tool role messages and code blocks with context-aware filters.

How do I configure a stacked pipeline using both RTK and Caveman engines?

Pass the stacked mode to applyCompression with a stackedPipeline array defining the execution order. Configure each engine separately via rtkConfig and cavemanConfig within the config object. The default fallback stack automatically applies RTK followed by Caveman when no explicit pipeline is defined but stacked mode is enabled.

Does the RTK engine support both OpenAI and Anthropic tool formats?

Yes. RTK recognizes both OpenAI-style tool roles with tool_call_id and Anthropic-style tool_use and tool_result blocks. The engine specifically preserves Anthropic's cache_control markers and outer block structure while compressing inner content, as implemented in the special handling section at lines 72–87 of engines/rtk/index.ts.

What protected structures does Caveman avoid modifying?

Caveman uses the PROTECTED_STRUCTURE_RE regex and hasProtectedStructure function to identify and preserve code fences (triple backticks), inline URLs, error stack traces, markdown tables, and other technical formatting. This protection ensures that compression rules do not corrupt structured data or code syntax during the normalization process.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}