How to Configure OmniRoute's 10-Engine Compression Pipeline for Optimal Token Savings

To achieve optimal token savings in OmniRoute, configure a stackedPipeline ordered as ["lite", "caveman", "rtk", "ultra"], set each engine's intensity to its maximum validated value (e.g., caveman: "ultra", rtk: "aggressive"), and set autoTriggerMode to "ultra" with autoTriggerTokens at 200 so the most aggressive reduction engages early on every request.

OmniRoute, the open-source routing layer maintained at diegosouzapw/OmniRoute, reduces upstream LLM costs by passing every request through a multi-engine prompt-compression pipeline before it reaches a provider. The pipeline's behavior is controlled by records in the key_value table, normalized in src/lib/db/compression.ts, and strictly validated against src/shared/validation/compressionConfigSchemas.ts. Understanding how to order these engines and dial their intensities is the fastest path to maximizing token savings.

High-Level Architecture of the OmniRoute 10-Engine Compression Pipeline

OmniRoute applies compression through discrete compression engines that each perform a specific reduction technique, such as whitespace collapse, semantic condensation, or RTK-based filtering. The available engine identifiers are hard-coded as a runtime catalog in src/lib/db/compression.ts so that persisted configurations never drift from what the server can execute.

// src/lib/db/compression.ts
const STACKED_PIPELINE_ENGINE_IDS = new Set([
  "lite", "caveman", "aggressive", "ultra", "rtk",
  "headroom", "session-dedup", "ccr", "llmlingua",
  "relevance", "omniglyph",
]);

Every engine declares its allowed intensities in src/shared/validation/compressionConfigSchemas.ts. The intensity determines how aggressively an engine rewrites or prunes content.

// src/shared/validation/compressionConfigSchemas.ts
export const STACKED_PIPELINE_ENGINE_INTENSITIES: Record<string, readonly string[]> = {
  "session-dedup": [],
  ccr: [],
  lite: ["lite"],
  rtk: ["minimal", "standard", "aggressive"],
  headroom: [],
  relevance: [],
  caveman: ["lite", "full", "ultra"],
  aggressive: ["standard", "ultra"],
  llmlingua: [],
  omniglyph: [],
  ultra: ["ultra"],
};

A request can be processed in two pipeline styles: a fallback defined by defaultMode or autoTriggerMode, or an explicit ordered list called stackedPipeline that is normalized by normalizeStackedPipeline() in src/lib/db/compression.ts.

Core Data Flow

When a chat request arrives, the handler chain in open-sse/handlers/chatCore/* resolves compression settings through a four-stage flow:

1. Read settings — getCompressionSettings() queries the key_value table, normalizes fields such as cavemanConfig and rtkConfig, and caches the result for five seconds. This logic lives in src/lib/db/compression.ts.

2. Derive the engine map — If an explicit engines row exists, it is used directly; otherwise the map is derived from legacy fields like defaultMode and combo defaults.

3. Build the stacked pipeline — normalizeStackedPipeline() filters out unknown engine IDs and returns a deterministic array that the runtime will traverse in order.

4. Execute each engine — The implementations in open-sse/services/compression/* apply the actual token-saving logic, streaming the request through each step sequentially.

Tuning the Pipeline for Maximum Token Savings

The following settings in the compression namespace control how aggressively OmniRoute reduces prompt size. Adjusting them together yields the highest token savings.

• autoTriggerMode / autoTriggerTokens — These control when the fallback pipeline runs automatically. Set autoTriggerMode to "ultra" and autoTriggerTokens to 200 so the most aggressive engine engages early, even on moderately sized prompts.

• cavemanConfig.intensity — The caveman engine performs semantic rewriting. Use intensity "ultra" for maximum condensation, but validate output quality because this rewrites natural language.

• rtkConfig.intensity — RTK filters tool output. An intensity of "aggressive" removes the most lines and delivers the biggest cutback when tool results are large.

• ultraConfig.compressionRate — This SLM-based engine targets a specific token percentage. A value of 0.7 tells the engine to compress down to roughly 30 percent of the original token count while preserving a quality fallback.

• stackedPipeline order — Engines run sequentially, so placing lighter engines first prunes content early and lets heavier engines work on a smaller prompt. The typical optimal order is ["lite", "caveman", "rtk", "ultra"].

• cacheMinutes — Reuses a compressed prompt for a short window. Keep this low (around 5) if prompts vary rapidly, or raise it to 30 for static workloads that repeat.

• preserveSystemPromptMode — Determines whether the system prompt is untouchable. Use "always" only when the system prompt carries essential instructions; otherwise "whenNoCache" avoids unnecessary token loss.

Updating the Compression Configuration

Via the REST API

You can patch the entire compression namespace in a single call. The endpoint delegates to updateCompressionSettings() inside src/lib/db/compression.ts.

curl -X PATCH https://<host>/api/settings/compression \
  -H "Authorization: Bearer <API-KEY>" \
  -H "Content-Type: application/json" \
  -d '{
        "enabled": true,
        "autoTriggerMode": "ultra",
        "autoTriggerTokens": 200,
        "cavemanConfig": { "enabled": true, "intensity": "ultra" },
        "rtkConfig": { "enabled": true, "intensity": "aggressive" },
        "ultra": { "enabled": true, "compressionRate": 0.7 },
        "stackedPipeline": [
          { "engine": "lite" },
          { "engine": "caveman", "intensity": "ultra" },
          { "engine": "rtk",     "intensity": "aggressive" },
          { "engine": "ultra",   "intensity": "ultra" }
        ]
      }'

Via the CLI

The CLI entry point at bin/cli/commands/compression.mjs supports string-based pipeline shorthand.

# Show current settings

omniroute compression get

# Set the optimal stacked pipeline

omniroute compression set \
  --pipeline "lite,caveman:ultra,rtk:aggressive,ultra:ultra"

# Enable auto-trigger with aggressive savings

omniroute compression set \
  --auto-trigger-mode ultra \
  --auto-trigger-tokens 200

Programmatic DB Updates

For programmatic control inside the codebase, import updateCompressionSettings from @/lib/db/compression. The helper runs inside a transaction, calls sanitizeEnginesForWrite, and clears the five-second cache automatically.

import { updateCompressionSettings } from "@/lib/db/compression";

await updateCompressionSettings({
  autoTriggerMode: "ultra",
  autoTriggerTokens: 200,
  cavemanConfig: { enabled: true, intensity: "ultra" },
  stackedPipeline: [
    { engine: "lite" },
    { engine: "caveman", intensity: "ultra" },
    { engine: "rtk", intensity: "aggressive" },
    { engine: "ultra", intensity: "ultra" },
  ],
});

Verifying Token Savings

After applying changes, measure real-world savings through the built-in telemetry endpoint backed by src/lib/db/compressionRunTelemetry.ts.

curl https://<host>/api/compression/run-telemetry \
  -H "Authorization: Bearer <API-KEY>"

The response returns originalTokens, compressedTokens, and the percentage saved for each run, letting you compare configurations precisely.

Summary

• OmniRoute's compression pipeline is defined by STACKED_PIPELINE_ENGINE_IDS in src/lib/db/compression.ts and validated by src/shared/validation/compressionConfigSchemas.ts.
• Use a stackedPipeline ordered as ["lite", "caveman", "rtk", "ultra"] for maximum sequential pruning.
• Set caveman to "ultra", rtk to "aggressive", and ultraConfig.compressionRate to 0.7 for aggressive reduction targets.
• Trigger compression early with autoTriggerMode: "ultra" and autoTriggerTokens: 200.
• Update settings through the REST API, CLI (bin/cli/commands/compression.mjs), or programmatically via updateCompressionSettings().
• Verify results with the /api/compression/run-telemetry endpoint.

Frequently Asked Questions

What is the optimal engine order for maximum token savings?

The recommended stackedPipeline order is ["lite", "caveman", "rtk", "ultra"] because it runs cheaper pruning engines first, reducing the token volume before the heavier SLM-based ultra engine executes. This ordering is enforced by normalizeStackedPipeline() in src/lib/db/compression.ts.

How do I enable the most aggressive compression intensity across all engines?

Set cavemanConfig.intensity to "ultra", rtkConfig.intensity to "aggressive", and include { engine: "ultra", intensity: "ultra" } in your stackedPipeline. Also set autoTriggerMode to "ultra" so the fallback path uses the strongest engine when no explicit pipeline is supplied.

Where are compression settings stored and cached?

All values live in the key_value table under the compression namespace. getCompressionSettings() in src/lib/db/compression.ts reads and normalizes these rows, then caches the result for five seconds. Calling updateCompressionSettings() invalidates that cache immediately.

How can I measure token savings after changing the pipeline?

Query the /api/compression/run-telemetry endpoint or inspect src/lib/db/compressionRunTelemetry.ts. The telemetry object reports originalTokens, compressedTokens, and the exact percentage saved per request, making it easy to A/B test different engine orders and intensities.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}