What Is the OmniRoute 10-Engine Compression Pipeline and How It Achieves Token Savings

The OmniRoute 10-engine compression pipeline is a configurable, multi-stage request processing system that reduces LLM prompt tokens by up to 45% through rule-based condensation, semantic summarization, and optional small language model pruning, while maintaining fail-open safety guards.

The OmniRoute 10-engine compression pipeline powers the request-handling layer of the open-source diegosouzapw/OmniRoute project, shrinking prompts before they reach upstream LLM providers. Implemented in the open-sse/services/compression package, this pipeline selects from multiple compression strategies—from lightweight whitespace removal to aggressive token pruning—to minimize API costs and latency without sacrificing semantic fidelity.

How the Compression Mode Selector Works

The entry point selectCompressionPlan() in [strategySelector.ts](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/strategySelector.ts) determines which compression mode to apply using a priority-ordered decision tree:

1. Master switch — If config.enabled === false, the pipeline returns "off"
2. Routing-combo override — comboOverrides can force a specific mode for designated route combinations
3. Header-driven profile — planFromHeader respects client-specified modes via request headers
4. Active named combo — config.activeComboId triggers "stacked" mode with a predefined engine sequence
5. Auto-trigger — Large prompts automatically activate config.autoTriggerMode
6. Derived default — Falls back to deriveDefaultPlan matching config.defaultMode

When a caching context is detected (detectCachingContext), the cache-aware guard (getCacheAwareStrategy) may adjust the selected mode. The final output is a DerivedPlan containing the effective mode and, for stacked configurations, the ordered stackedPipeline.

The Six Compression Engines and Token-Saving Mechanisms

Once selected, applyCompression() (strategySelector.ts#L52) dispatches to specialized engines. Each engine produces a CompressionResult containing the transformed body, a compressed flag, and CompressionStats (defined in [types.ts](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/types.ts)).

Lite Mode: Whitespace and Duplicate Removal

applyLiteCompression in [lite.ts](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/lite.ts) performs simple whitespace normalization and duplicate system-prompt removal. This yields approximately 10% token savings with virtually zero latency overhead.

Standard Mode: Caveman Rule-Based Condensation

cavemanCompress in [caveman.ts](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/caveman.ts) applies regex-driven semantic rules that strip filler phrases, deduplicate system content, and compress verbose user messages. Typical savings range from 15–30% depending on input verbosity.

Aggressive Mode: Summarization and Aging

compressAggressive in [aggressive.ts](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/aggressive.ts) targets long conversation histories by summarizing older messages, truncating tool results, and progressively degrading aging turns based on configurable thresholds. This dramatically reduces token counts for stateful chat sessions.

Ultra Mode: Heuristic and SLM Tiered Pruning

ultraCompress in [ultra.ts](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/ultra.ts) implements a two-tier strategy:

• Tier-A (Heuristic): ultraCompressHeuristic (ultra.ts#L33) scores tokens and discards the lowest-scoring half based on compressionRate
• Tier-B (SLM): Runs a local ONNX model (llmlingua) for model-aware pruning; falls back to Tier-A if the SLM is unavailable

This combination achieves deep compression for high-volume traffic.

RTK Mode: Tool Output Filtering

applyRtkCompression in [rtk/index.ts](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/engines/rtk/index.ts) filters low-value lines from tool results (e.g., git diffs, shell outputs) and deduplicates repeated command outputs. This mode is essential for agentic workflows generating verbose telemetry.

Stacked Mode: Multi-Engine Pipelines

applyStackedCompression executes a configurable sequence of engines (e.g., rtk → caveman → ultra) orchestrated by [stackedStepCore.ts](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/stackedStepCore.ts). Each step may compress further, with early bail-out if gains become marginal. The guardPipelineInflation safety mechanism ensures the pipeline never returns a larger payload than the original request.

Safety Guards and Hard Budget Controls

The pipeline implements multiple fail-open safety guards to protect production traffic:

• Pipeline inflation guard (guardPipelineInflation in [pipelineGuards.ts](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/pipelineGuards.ts)): Reverts to the original body if any step increases token count
• Circuit breaker (pipelineEngineBreaker.ts): Skips steps that have repeatedly failed
• Fidelity gate (fidelityGateStep.ts): Enforces minimum quality thresholds before accepting step output
• Risk gate (riskGate/): Masks sensitive spans (PII) before compression to prevent data loss

After all engines complete, applyHardBudget in [hardBudget.ts](https://github.com/diegosouzapw/OmniRoute/blob/main/open-sse/services/compression/hardBudget.ts) performs a deterministic trim to enforce targetTokens or targetRatio limits.

Measuring Token Savings with CL100K

All engines compute token counts using the CL100K tokenizer (OpenAI's tiktoken standard). The CompressionStats object records:

• originalTokens: Count before processing
• compressedTokens: Count after processing
• Compression percentage: 100 × (1 - compressedTokens / originalTokens)

Telemetry persists via the compression DB module (src/lib/db/compression.ts) for dashboard reporting. In practice, a typical 1,200-token chat request processed through a ["rtk","caveman","ultra"] stacked pipeline often reduces to 650–800 tokens, achieving 35–45% reduction while preserving semantic fidelity.

Practical Implementation Examples

Basic Route Integration

import { applyCompression } from '@/open-sse/services/compression/strategySelector';
import { getCompressionSettings } from '@/src/lib/db/compression';

export async function POST(req: Request) {
  const body = await req.json();
  const config = await getCompressionSettings();
  
  const mode = selectCompressionStrategy(
    config,
    null,
    estimateTokens(body),
    body
  );
  
  const { body: compressedBody, compressed, stats } = applyCompression(
    body,
    mode,
    { config, principalId: req.headers.get('x-omniroute-principal') }
  );
  
  // compressedBody contains the optimized payload
  const upstreamResult = await executor.execute(compressedBody);
}

Custom Stacked Pipeline

import { applyStackedCompression } from '@/open-sse/services/compression/strategySelector';
import type { CompressionPipelineStep } from '@/open-sse/services/compression/types';

const myPipeline: CompressionPipelineStep[] = [
  { engine: 'rtk', intensity: 'standard' },
  { engine: 'caveman', intensity: 'full' },
  { engine: 'ultra', intensity: 'standard' },
];

const result = applyStackedCompression(originalBody, myPipeline, {
  config: myCompressionConfig,
  onEngineStep: (step) => console.log('Engine step', step),
});

// result.stats contains per-engine breakdown

Async Ultra with SLM Fallback

import { applyCompressionAsync } from '@/open-sse/services/compression/strategySelector';

const result = await applyCompressionAsync(
  originalBody,
  'ultra',
  {
    config: {
      ...myCompressionConfig,
      ultraEngine: 'slm',
      ultra: { modelPath: '/opt/models/llmlingua.onnx' },
    },
  }
);

// stats.ultraTier indicates "slm" or "heuristic-fallback"

Summary

• The OmniRoute 10-engine compression pipeline provides six distinct processing modes (off, lite, standard, aggressive, ultra, rtk) plus configurable stacked combinations
• Token savings range from 10% (lite) to 45% (stacked aggressive/ultra) depending on input characteristics and selected engines
• Safety guards including inflation checks, circuit breakers, and hard budgets ensure the pipeline fails open to the original request if any step underperforms
• All measurements use the CL100K tokenizer for consistency with OpenAI billing
• Configuration supports both synchronous and asynchronous execution, with optional local SLM (LLMLingua) integration for ultra mode

Frequently Asked Questions

What happens if a compression step increases the token count instead of decreasing it?

The guardPipelineInflation mechanism in pipelineGuards.ts automatically detects when any engine step produces a larger payload than its input. When this occurs, the pipeline discards that step's output and reverts to the previous valid state—or the original request body if no valid state exists—ensuring the pipeline never harms token efficiency.

Can I use the compression pipeline without the local SLM (LLMLingua) model?

Yes. The ultra mode implements a two-tier architecture. If the ONNX model (llmlingua) is unavailable or ultraEngine is set to "heuristic", the system automatically falls back to the heuristic tier (ultraCompressHeuristic), which uses token-scoring algorithms without requiring external model dependencies.

How does the hard budget enforcer interact with the stacked pipeline?

The applyHardBudget function in hardBudget.ts runs as a post-pass after all stacked engines complete. It performs a deterministic trim to ensure the final payload respects targetTokens or targetRatio limits, acting as a final safety net even if the cumulative pipeline compression falls short of requirements.

Which tokenizer does OmniRoute use for calculating token savings?

The pipeline uses the CL100K tokenizer (the same encoding used by OpenAI's tiktoken library) to calculate originalTokens and compressedTokens in the CompressionStats object. This ensures that reported savings percentages accurately reflect the actual billing impact when using OpenAI-compatible providers.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}