Prompt Compression Pipeline Architecture in OmniRoute: RTK and Caveman Modes Explained

OmniRoute's prompt compression pipeline uses a modular, mode-based architecture where RTK applies rule-driven terminal output processing and Caveman performs aggressive semantic condensation, both orchestrated through a strategy selector that dispatches to dedicated engine registries.

The prompt compression system in diegosouzapw/OmniRoute lives under open-sse/services/compression/ and shrinks prompts before upstream transmission. This article breaks down the prompt compression pipeline architecture for its two specialized modes—RTK (Rule-Based Terminal-Kit) and Caveman—showing how requests flow from mode selection through engine dispatch to final compressed output.

How Requests Enter the Prompt Compression Pipeline

Every compressed prompt follows a two-stage entry process governed by configurable logic.

Mode Selection via strategySelector.ts

The strategySelector.ts module determines which CompressionMode to activate. It evaluates:

• Combo-override settings via checkComboOverride()
• Auto-trigger thresholds via shouldAutoTrigger()
• Assigned compression combo, default fallback, and cache-aware configuration

The resolved mode—rtk, caveman, lite, standard, aggressive, ultra, off, or stacked—is returned as a concrete CompressionMode enum value.

Source: [open-sse/services/compression/strategySelector.ts](https://github.com/diegosouzapw/OmniRoute/blob/release/v3.8.50/open-sse/services/compression/strategySelector.ts)

Engine Dispatch via Registry

The selected mode maps to an engine implementation in engines/registry.ts:

Mode	Engine Entry Point
rtk	engines/rtk/index.ts
caveman	engines/cavemanAdapter.ts

Source: [open-sse/services/compression/engines/registry.ts](https://github.com/diegosouzapw/OmniRoute/blob/release/v3.8.50/open-sse/services/compression/engines/registry.ts)

RTK Mode: Rule-Based Terminal-Kit Architecture

RTK is a highly configurable, rule-driven compressor optimized for command-line output—tool results, shell diffs, Terraform plans, and structured terminal data. Its architecture implements a pipeline of independent, stackable steps.

Core RTK Pipeline Components

Component	Responsibility	Key File
Command detection	Identifies blocks resembling commands or tool invocations	engines/rtk/commandDetector.ts
Deduplication	Removes repeated lines across entire output	engines/rtk/deduplicator.ts
Line filtering	Strips comments, ANSI codes, and noise	engines/rtk/lineFilter.ts
Smart truncate	Truncates long outputs while preserving essential context	engines/rtk/smartTruncate.ts
Renderers	Converts structured data (Terraform plans, git diffs, tables) to concise text	engines/rtk/renderers/*
Config schema & loader	Validates RTK configuration and loads filter modules	engines/rtk/configSchema.ts, engines/rtk/filterLoader.ts
Learning / caching	Optional cache-aware processing for remembered truncation decisions	engines/rtk/learn.ts
Verification	Safety guard test harness (no-redos assurance)	engines/rtk/verify.ts

RTK Execution Model

The engines/rtk/index.ts module assembles these steps into an RTKPipeline based on resolved configuration. It exposes two execution paths:

• applyCompression — synchronous execution
• applyCompressionAsync — asynchronous execution with non-blocking I/O

import { applyCompression } from '@/open-sse/services/compression/engines/rtk';

const rtkConfig = {
  steps: ['commandDetector', 'deduplicator', 'smartTruncate'],
  maxLines: 200,
  preserveHeaders: true
};

const input = { text: largeTerraformOutput };
const { compressed, stats } = applyCompression(rtkConfig, input);

console.log(`RTK saved ${stats.savingsPct}% token budget`);

Source: [open-sse/services/compression/engines/rtk/index.ts](https://github.com/diegosouzapw/OmniRoute/blob/release/v3.8.50/open-sse/services/compression/engines/rtk/index.ts)

Caveman Mode: Semantic Condensation Architecture

Caveman is a semantic, rule-based condenser that aggressively shrinks whitespace and duplicate sections while preserving code meaning. It trades configurability for compression ratio.

Caveman Architecture Layers

Layer	Component	Responsibility
Core engine	caveman.ts	Implements pattern-matching + heuristic condensation algorithm
Rule set	cavemanRules.ts	Hand-crafted rules: dedupSystemPrompt, collapseWhitespace, stripRedundantImports
Adapter	cavemanAdapter.ts	Bridges generic pipeline API to Caveman core; exposes applyCompression()

When strategySelector.ts returns caveman, the adapter loads the relevant rule subset, executes the core engine, and returns transformed text through the standard engine interface.

import { applyCompression } from '@/open-sse/services/compression/engines/cavemanAdapter';

const cavemanConfig = {
  rules: ['dedupSystemPrompt', 'collapseWhitespace', 'compactJSON'],
  aggressive: true
};

const { compressed, stats } = applyCompression(cavemanConfig, {
  text: verboseSystemPrompt
});

Sources: [caveman.ts](https://github.com/diegosouzapw/OmniRoute/blob/release/v3.8.50/open-sse/services/compression/caveman.ts), [cavemanRules.ts](https://github.com/diegosouzapw/OmniRoute/blob/release/v3.8.50/open-sse/services/compression/cavemanRules.ts), [cavemanAdapter.ts](https://github.com/diegosouzapw/OmniRoute/blob/release/v3.8.50/open-sse/services/compression/engines/cavemanAdapter.ts)

Stacked and Hybrid Pipelines

The prompt compression pipeline architecture supports combining multiple engines through stackedStepCore.ts. When enginesMapDerivesStackedPipeline() returns true, engines execute sequentially with each output feeding the next stage.

Common stacking patterns:

• rtk → caveman — Terminal output first filtered structurally, then semantically condensed
• lite → standard — Progressive compression tiers

stats.ts aggregates token-savings metrics across all stages for monitoring and API exposure.

Source: [open-sse/services/compression/stackedStepCore.ts](https://github.com/diegosouzapw/OmniRoute/blob/release/v3.8.50/open-sse/services/compression/stackedStepCore.ts)

End-to-End Request Flow

Incoming SSE request
    │
    └─► strategySelector.selectCompressionPlan(config, estimatedTokens)
           │
           ├─► Mode: 'rtk'
           │       └─► RTKPipeline.execute()
           │               ├─ commandDetector → deduplicator
           │               ├─ lineFilter → smartTruncate
           │               └─ renderers[*] → output
           │
           ├─► Mode: 'caveman'
           │       └─► cavemanAdapter
           │               └─ caveman core + cavemanRules[config.rules]
           │
           └─► Mode: 'stacked'
                   └─► stackedStepCore
                           ├─ engine[0] → engine[1] → ...
                           └─ stats.aggregate()

    ◄───── compressed prompt + savings metrics
                ↓
        chatCore → upstream LLM provider

Metrics collection via stats.ts records original tokens, compressed tokens, per-engine savings, and compression ratio. These feed the Compression API at src/app/api/settings/compression/.

Source: [open-sse/services/compression/stats.ts](https://github.com/diegosouzapw/OmniRoute/blob/release/v3.8.50/open-sse/services/compression/stats.ts)

Mode Selection in Practice

import { selectCompressionPlan } from '@/open-sse/services/compression/strategySelector';

// Typical integration point in SSE handler
const plan = selectCompressionPlan(
  request.compressionConfig,  // User/org settings
  estimateTokens(promptText)  // Heuristic or tiktoken-based
);

// plan.mode: 'rtk' | 'caveman' | 'lite' | 'standard' | 'aggressive' | 'ultra' | 'off' | 'stacked'
// plan.engineConfig: mode-specific parameters
// plan.stack: ordered engine array (if stacked)

Source: [strategySelector.ts](https://github.com/diegosouzapw/OmniRoute/blob/release/v3.8.50/open-sse/services/compression/strategySelector.ts)

Comparing RTK and Caveman Modes

Characteristic	RTK Mode	Caveman Mode
Primary target	Terminal output, command results, structured logs	Code blocks, system prompts, repetitive text
Configurability	High (JSON schema, custom filters, renderers)	Low (predefined rule selection)
Processing steps	Multi-stage pipeline (detect → filter → truncate → render)	Single-pass semantic condensation
Extensibility	Custom renderers via renderers/*	Fixed rule set in cavemanRules.ts
Safety guarantees	verify.ts test harness	Implicit in rule design
Best for	Terraform plans, git diffs, build logs, tables	System prompts, API schemas, boilerplate reduction

Summary

• OmniRoute's prompt compression pipeline architecture lives in open-sse/services/compression/ with mode selection via strategySelector.ts and engine dispatch via engines/registry.ts.

• RTK mode implements a configurable, multi-step pipeline for terminal-style output with components for command detection, deduplication, filtering, truncation, and structured rendering.

• Caveman mode provides aggressive semantic condensation through pattern matching and heuristic rules, accessed via cavemanAdapter.ts with core logic in caveman.ts.

• Stacked pipelines combine engines sequentially through stackedStepCore.ts, enabling hybrid compression strategies with aggregated metrics via stats.ts.

• Both engines expose uniform applyCompression() / applyCompressionAsync() APIs for seamless integration with the SSE request handler and upstream LLM providers.

Frequently Asked Questions

How does OmniRoute decide between RTK and Caveman modes?

The strategySelector.ts module evaluates combo-override settings, auto-trigger thresholds, assigned compression combos, and cache-aware configuration to return a concrete CompressionMode. User settings take precedence; when unspecified, auto-trigger logic estimates token savings potential and selects the appropriate mode.

Can RTK and Caveman be used together?

Yes. The stacked mode orchestrates sequential engine execution via stackedStepCore.ts. A common pattern runs RTK first for structural terminal processing, then Caveman for final semantic condensation. Each engine's output feeds the next, with stats.ts aggregating cumulative savings.

What file types or content does RTK handle best?

RTK excels with command-line outputs: Terraform plans (engines/rtk/renderers/terraformPlan.ts), git diffs (engines/rtk/renderers/gitDiff.ts), tabular data, and ANSI-colored terminal logs. Its renderer architecture allows adding custom parsers for new tool formats without modifying core pipeline logic.

Is Caveman mode reversible or lossless?

No—Caveman is intentionally lossy. It applies aggressive heuristics from cavemanRules.ts (whitespace collapse, duplicate section removal, import deduplication) that discard formatting to preserve semantic meaning. For reversible compression, use lite or standard modes instead.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}