OmniRoute Compression Engines: Supported Types and Pipeline Architecture

TLDR: OmniRoute supports ten distinct compression engines—including Lite, Caveman, RTK, and LLMLingua—that are chained through a deterministic pipeline involving strategy selection, guardrail checks, and sequential execution via stackedStepCore.ts.

The OmniRoute inference router implements a sophisticated prompt-compression subsystem under open-sse/services/compression to minimize token costs before dispatching requests to LLM providers. This article maps every supported compression engine and explains how the system pipelines them through registry-based orchestration and risk-aware guardrails as implemented in the diegosouzapw/OmniRoute source.

Supported Compression Engines in OmniRoute

Lite Engine

Located in open-sse/services/compression/lite.ts, the Lite engine provides fast, low-overhead compression using five deterministic tricks: whitespace collapse, system-prompt deduplication, tool-result compression, redundant-content removal, and image-URL replacement. This engine serves as the default for "lite" mode requests where latency matters more than semantic depth.

Caveman Engine

The Caveman engine, found in open-sse/services/compression/engines/caveman/*, applies rule-based semantic condensation using language-pack rule files to strip verbose phrasing while preserving meaning. It targets prompts requiring moderate compression without external model dependencies.

RTK (Rule-Based Terminal/Tool-Output Kernel)

Residing in open-sse/services/compression/engines/rtk/*, the RTK engine parses command-line output, filters JSON structures, trims lines, and renders specialized formats like Terraform plans or Git diffs through a sophisticated rule-DSL. This engine excels at compressing structured tool outputs into token-efficient summaries.

Relevance Engine

Implemented in open-sse/services/compression/engines/relevance/*, the Relevance engine scores each prompt chunk for contextual importance and surgically trims low-impact sections. It uses heuristic-based scoring rather than lexical rules to maintain conversational coherence while reducing token count.

Session-Dedup Engine

The Session-Dedup engine in open-sse/services/compression/engines/session-dedup/* employs fuzzy matching to eliminate repeated content across the current session context. This prevents token waste from redundant system instructions or repeated code blocks in multi-turn conversations.

MCP Accessibility Engine

Located in open-sse/services/compression/engines/mcpAccessibility/*, this specialized engine collapses repeated MCP (Model Context Protocol) tool output to improve streaming performance. It specifically targets the verbosity patterns common in MCP-based tool integrations.

LLMLingua Adapter

The LLMLingua adapter in open-sse/services/compression/engines/llmlingua/* provides a bridge to the external llmlingua library for token-aware summarization. This option leverages external ML models when aggressive compression is required and latency constraints permit.

Experimental and Budget-Aware Engines

OmniRoute includes three specialized engines for edge cases: the Omniglyph engines (omniglyphSingleMode.ts and omniglyphAdapter.ts) for experimental aggressive compression; the Quantum-Lock engine (quantumLock/*) acting as a risk-aware budget-gate that aborts compression when token budgets are exceeded; and Progressive Aging (progressiveAging.ts) which gradually relaxes compression aggressiveness over time to balance quality and cost.

How Compression Engines Are Pipelined

Strategy Selection via strategySelector.ts

The pipeline begins in open-sse/services/compression/strategySelector.ts, which examines the request's compression mode (off, lite, standard, aggressive, ultra, rtk, stacked) alongside compression combo assignments and auto-trigger thresholds. This module returns a deterministic plan specifying which engines execute and in what order.

Engine Registry and Pipeline Construction

The engine registry (open-sse/services/compression/engines/registry.ts) maintains a mapping of string identifiers (e.g., "rtk", "caveman") to concrete implementation classes. For stacked modes, the registry instantiates multiple engines in the sequence defined by the strategy plan, creating a chain where each engine's output feeds the next input.

Guardrails and Budget Gates via pipelineGuards.ts

Before execution, open-sse/services/compression/pipelineGuards.ts applies risk-gate and fidelity-gate validations. These guardrails can short-circuit the entire pipeline or skip specific engines if token-budget thresholds or quality constraints are violated, preventing over-compression of critical prompts.

Sequential Execution via stackedStepCore.ts

The stackedStepCore.ts module orchestrates the actual execution, iterating through the ordered engine list and invoking each engine's compress(CompressionContext) method. Each engine receives a CompressionContext object containing the current prompt and token budget, returning a modified prompt that propagates downstream.

Memoization and Telemetry

After execution, resultMemo.ts caches intermediate results to avoid re-running pipelines for identical prompts, while stats.ts records token counts, savings percentages, and applied engines for dashboard visibility. This telemetry feeds the UI and internal compression-combo analytics available through the public API in open-sse/services/compression/index.ts.

Controlling Compression Engines in Code

Setting Compression Mode via API

Developers trigger compression by resolving a plan through resolveCompressionPlan.ts:

// In a Next.js API route (simplified)
import { resolveCompressionPlan } from '@omniroute/open-sse/services/compression/resolveCompressionPlan';

export async function GET(req) {
  const url = new URL(req.url);
  const mode = url.searchParams.get('compression') ?? 'off'; // e.g. "lite", "rtk", "stacked"

  // Resolve the plan based on the mode and any assigned combo
  const plan = await resolveCompressionPlan({ 
    compressionMode: mode, 
    requestId: req.headers.get('x-request-id') 
  });

  // The plan attaches to context; downstream handlers run it automatically
  return handleChat(req, { compressionPlan: plan });
}

Manual Engine Invocation for Testing

Individual engines can be instantiated directly from the registry for unit testing:

import { getEngine } from '@omniroute/open-sse/services/compression/engines/registry';
import { CompressionContext } from '@omniroute/open-sse/services/compression/types';

async function testRtkEngine(prompt: string) {
  const ctx: CompressionContext = { prompt, tokenBudget: 4096 };
  const rtk = getEngine('rtk');
  const result = await rtk.compress(ctx);
  console.log('Compressed prompt:', result.prompt);
}

Inspecting Compression Statistics

Post-request metrics are available through the stats module:

import { getStats } from '@omniroute/open-sse/services/compression/stats';

async function logStats(requestId: string) {
  const stats = await getStats(requestId);
  console.log(`Original tokens: ${stats.originalTokens}`);
  console.log(`Compressed tokens: ${stats.compressedTokens}`);
  console.log(`Savings: ${(stats.savingsPct * 100).toFixed(1)}%`);
  console.log(`Engines applied: ${stats.engines.join(', ')}`);
}

Summary

• OmniRoute provides ten compression engines ranging from fast rule-based options (Lite, Caveman) to sophisticated semantic processors (RTK, Relevance, LLMLingua).
• The pipeline architecture in open-sse/services/compression follows a deterministic lifecycle: strategy selection → guardrail validation → registry-based assembly → sequential execution via stackedStepCore.ts.
• Risk-aware guardrails in pipelineGuards.ts enforce token budgets and fidelity thresholds, preventing destructive over-compression.
• Memoization and telemetry systems optimize performance and provide visibility into token savings per request.

Frequently Asked Questions

Which OmniRoute compression engine offers the fastest performance?

The Lite engine in open-sse/services/compression/lite.ts delivers the lowest latency by applying five deterministic tricks—whitespace collapse, system-prompt deduplication, and image-URL replacement—without invoking external models or complex parsers. It is optimized for high-throughput scenarios where millisecond-level overhead is critical.

How does the RTK engine differ from the Caveman engine?

While Caveman uses language-pack rule files for general semantic condensation, the RTK (Rule-Based Terminal/Tool-Output Kernel) engine implements a sophisticated rule-DSL specifically designed for structured data like JSON, Terraform plans, and Git diffs. RTK resides in open-sse/services/compression/engines/rtk/* and performs deep parsing of command-line output, whereas Caveman applies broader linguistic rules across unstructured text.

Can multiple compression engines run on a single request?

Yes. The stacked compression mode chains multiple engines sequentially through stackedStepCore.ts, feeding the output of one engine as input to the next. The order is determined by strategySelector.ts and enforced by the registry in engines/registry.ts, allowing combinations like Lite → RTK → Relevance for aggressive compression scenarios.

What prevents OmniRoute from over-compressing critical prompts?

The Quantum-Lock engine (quantumLock/*) acts as a budget-gate that aborts compression when token thresholds would be exceeded, while pipelineGuards.ts applies risk-gate and fidelity-gate validations before and during execution. These guardrails short-circuit the pipeline if compression would violate quality constraints or budget limits.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}