OmniRoute Compression Modes Explained: RTK, Caveman, Lite, Ultra, and Stacked

OmniRoute provides six distinct compression modes—off, lite, standard (Caveman), rtk, ultra, and stacked—that trade off latency against token reduction, ranging from zero-overhead bypass to aggressive multi-engine pipelines.

The diegosouzapw/OmniRoute repository implements a flexible prompt-compression system governed by the CompressionMode type defined in open-sse/services/compression/types.ts. Selecting the correct OmniRoute compression mode directly impacts inference costs, latency, and prompt fidelity, making it critical for high-throughput AI routing workloads.

Understanding the CompressionMode Type System

At the core of the pipeline lies the CompressionMode union type exported from open-sse/services/compression/types.ts. This type enumerates the available engines: off, lite, standard, caveman, rtk, ultra, and stacked. Each mode maps to a specific engine implementation that transforms the incoming prompt before upstream transmission.

The valid modes are enforced by the COMPRESSION_MODES constant set in src/lib/db/compression.ts, which validates configuration values at runtime:

// src/lib/db/compression.ts
export const COMPRESSION_MODES = new Set([
  'off',
  'lite', 
  'standard',
  'caveman',
  'rtk',
  'ultra',
  'stacked'
]);

Off Mode: Zero-Overhead Debugging

off bypasses the compression pipeline entirely, guaranteeing that the original prompt is transmitted unchanged to the upstream provider.

Use this mode when debugging provider-specific formatting issues, validating prompt templates, or when you must preserve exact byte-for-byte fidelity for compliance reasons. It adds zero milliseconds of overhead and zero risk of content alteration.

Lite Mode: High-Speed Heuristics

lite invokes the engine defined in open-sse/services/compression/lite.ts, which applies five ultra-fast heuristic transformations:

• Whitespace collapse – Removes redundant spacing and newline sequences
• System-prompt deduplication – Eliminates duplicate system instructions
• Tool-result trimming – Truncates excessively long tool outputs to configurable limits
• Redundant-content removal – Strips repeated substrings across messages
• Image-URL replacement – Substitutes base64-heavy image strings with lightweight references

This mode adds less than 1 ms of latency and typically achieves 10–15 % token reduction. It remains fully deterministic, making it ideal for high-throughput workloads where speed is paramount and modest savings are sufficient.

Standard (Caveman) Mode: Balanced Semantic Condensing

standard (internally aliased as caveman) executes the rule-based engine in open-sse/services/compression/caveman.ts. This implementation applies a richer set of linguistic rules and language-pack heuristics than Lite, performing semantic condensation without natural-language generation.

Expect moderate processing time (a few milliseconds) and stronger token reductions of approximately 20–30 %. The output is deterministic for a given language pack, making this the default choice for general-purpose routing where you need a balance between speed and compression depth.

RTK Mode: Terminal and Tool-Output Optimization

rtk activates the specialized engine in open-sse/services/compression/rtk.ts, designed specifically for command-line and tool-output heavy prompts.

The RTK engine aggressively strips ANSI color codes, deduplicates repeated terminal lines, and applies JSON-focused filters to structured logs. Use this mode when routing prompts containing heavy shell transcripts, code execution output, or structured tool responses where preserving error details matters less than cutting noise.

Ultra Mode: Maximum Token Reduction

ultra triggers the aggressive pipeline defined in open-sse/services/compression/ultra.ts. This mode chains multiple engines—including the RTK engine—and applies aggressive semantic summarization that may rephrase content to minimize token count.

Expect the highest latency (tens of milliseconds) but significant savings of 40–50 % on token usage. Note that ultra mode may introduce non-determinism when vision-dependent steps are involved. Deploy this mode for very large prompts where cost reduction outweighs latency concerns and you can tolerate potential content loss.

Stacked Mode: Custom Engine Composition

stacked enables composite processing that runs a user-specified custom engine alongside the built-in OmniRoute pipeline. Implemented as a fallback resolver in open-sse/services/compression/strategySelector.ts, this mode checks for an engineId or custom pipeline configuration before delegating to the base mode.

Use stacked mode when you have proprietary compression logic (for example, a domain-specific summarizer) but still want the safety net of OmniRoute’s built-in heuristics. The strategy selector orchestrates the custom engine first, then falls back to the standard pipeline if the custom engine returns null or exceeds latency thresholds.

How OmniRoute Selects Compression Modes

The strategy selector (open-sse/services/compression/strategySelector.ts) resolves the final mode for each request through a three-tier precedence system:

1. Explicit request parameter – API callers can force a mode via the query string handled in src/app/api/compression/preview/route.ts
2. Combo-level fallback – If the request matches a combo configuration with fallbackCompressionMode set (stored in src/lib/db/compression.ts), OmniRoute applies that mode when the token count exceeds the combo’s threshold
3. Auto-trigger activation – The autoTriggerMode field automatically enables a specified mode once a request crosses the configured token budget

Configuration example for enabling proactive fallback:

// Configuration snippet referencing src/lib/db/compression.ts structures
const comboConfig = {
  id: 'production-api',
  fallbackCompressionMode: 'ultra',
  fallbackThreshold: 4000, // tokens
  autoTriggerMode: 'lite'
};

Previewing Compression Results

You can validate the output of any mode before applying it globally by calling the preview endpoint defined in src/app/api/compression/preview/route.ts:

curl -X POST "https://your-omniroute-instance/api/compression/preview?mode=ultra" \
  -H "Content-Type: application/json" \
  -d '{
    "prompt": "System: You are a helpful assistant\nUser: [Very long terminal output with ANSI codes...]"
  }'

The endpoint returns the transformed prompt and metadata including estimated token savings, allowing you to benchmark modes without affecting live traffic.

Summary

• off preserves original prompts exactly for debugging and compliance scenarios
• lite delivers sub-millisecond latency with 10–15 % token savings via five deterministic heuristics
• standard (Caveman) provides balanced 20–30 % reduction using rule-based semantic condensing
• rtk optimizes terminal and tool-output heavy prompts by stripping ANSI codes and deduplicating lines
• ultra maximizes cost savings at 40–50 % token reduction but adds tens of milliseconds of latency
• stacked composes custom engines with built-in pipelines for advanced hybrid compression strategies

Frequently Asked Questions

What is the difference between Caveman and Standard mode?

There is no functional difference; caveman is the legacy alias for standard. Both reference the same rule-based engine in open-sse/services/compression/caveman.ts. The caveman identifier exists for backward compatibility with older configuration files, while standard is the preferred modern designation.

When should I use RTK compression instead of Ultra?

Use RTK when your prompts contain heavy command-line output or structured tool logs that need noise reduction without aggressive rephrasing. RTK preserves semantic structure while stripping formatting artifacts. Ultra mode is superior for natural-language heavy contexts where you need maximum token slashing and can tolerate semantic rephrasing.

How do I completely disable compression in OmniRoute?

Set the compression mode to off in your combo configuration or API request. This bypasses the entire pipeline defined in open-sse/services/compression/strategySelector.ts, ensuring zero transformation overhead and guaranteeing prompt fidelity. Verify the setting via the preview endpoint at src/app/api/compression/preview/route.ts to confirm the output matches your input exactly.

Can I combine a custom compression engine with Lite mode?

Yes, by using stacked mode. Configure your custom engineId in the request metadata; the strategy selector will execute your custom engine first, then fall back to the lite pipeline if your engine returns null or times out. This composition is handled transparently by the resolution logic in open-sse/services/compression/strategySelector.ts.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}