What Is Token Compression in OmniRoute? A Complete Technical Guide

Token compression in OmniRoute is a configurable pipeline that reduces LLM token usage by applying compression engines to request and response payloads, cutting costs while preserving output quality.

OmniRoute implements token compression as a first-class architectural feature designed to minimize the number of tokens sent to and received from upstream LLM providers. This system intercepts requests after validation, applies pluggable compression engines, and transparently decompresses responses before they reach the client—all while maintaining full observability through built-in telemetry.

How Token Compression Works in OmniRoute

The compression pipeline integrates early in the request lifecycle. After Zod validation of the API body but before translation to provider-specific formats, OmniRoute evaluates compression configuration and prepares the payload for transformation.

The Core Compression Flow

When a request includes a compression object, OmniRoute executes four sequential steps:

1. Validates the configuration against src/shared/validation/compressionConfigSchemas.ts.
2. Stores the chosen engine in request context for downstream access.
3. Injects the x-omniroute-compression header with the engine identifier.
4. Dispatches to the provider and automatically decompresses the response using the matching engine.

The compression header serves as the coordination mechanism between OmniRoute and provider-side implementations. Providers or built-in transformers detect this header and return compact representations rather than full text.

Key Components of the Compression Pipeline

OmniRoute's token compression is built from six interconnected components, each with a dedicated responsibility:

• Compression Config Schemas — Located in src/shared/validation/compressionConfigSchemas.ts, this Zod-based validation layer ensures all compression requests specify valid engines and parameters.

• Compression Header Echo — The utility at src/shared/utils/compressionHeaderEcho.ts mirrors compression decisions back to clients, enabling telemetry integration and debugging visibility.

• Compression Engine Registry — Database mappings in src/lib/db/migrations/102_compression_engines_map.sql associate identifiers like caveman or stacked with concrete transformer implementations.

• MCP Compression Worker — Background worker pools handle token-reduction logic for both streaming and batch request patterns. Budget compliance is validated via scripts/check/check-compression-budget.ts.

• Per-API-Key Controls — Migration 150_api_key_compression_enabled.sql adds database-level opt-in flags, allowing operators to enable compression organization-wide or restrict it to specific keys.

• CLI Management Interface — The bin/cli/commands/compression.mjs module provides operator commands for runtime configuration changes.

Enabling Token Compression: Practical Examples

REST API Request with Compression

Include a compression object in your chat completion request to activate token reduction:

POST /v1/chat/completions HTTP/1.1
Content-Type: application/json
Authorization: Bearer <API_KEY>

{
  "model": "gpt-4o",
  "messages": [{ "role": "user", "content": "Explain quantum entanglement." }],
  "compression": { "engine": "caveman", "enabled": true }
}

The engine field references a registered compression algorithm. Available engines are defined in the migration 102_compression_engines_map.sql and loaded into the registry at runtime.

Global and Per-Key CLI Controls

Manage compression settings without code changes using the omniroute compression command:

# Enable compression for all API keys globally

omniroute compression --enable

# Disable compression for a specific key

omniroute compression --disable --key abc123

Telemetry Inspection

Analyze token savings and engine performance with the stats subcommand:

omniroute compression stats --key abc123

This surfaces tokens saved, compression engine utilization, and error rates from the compressionRunTelemetry and compressionAnalytics tables.

Observability and Cost Control

OmniRoute's token compression implementation emphasizes operational visibility. The system records granular telemetry in dedicated database tables, enabling:

• Budget compliance tracking — The scripts/check/check-compression-budget.ts quality-gate script validates that compression savings meet organizational targets.
• Engine performance comparison — Analytics tables store per-request metadata allowing A/B testing of caveman versus stacked engines.
• Error attribution — Failed decompressions are logged with full context for debugging.

Compression remains opt-in at the API-key level by default. This design prevents unexpected behavioral changes while giving operators precise control over which workloads incur the (typically minimal) latency overhead of transformation.

Summary

• Token compression in OmniRoute reduces LLM token counts through configurable, provider-integrated transformation engines.
• The pipeline activates post-validation via src/shared/validation/compressionConfigSchemas.ts and coordinates through the x-omniroute-compression header.
• Engine registry in migration 102_compression_engines_map.sql and per-key controls in migration 150_api_key_compression_enabled.sql enable flexible deployment patterns.
• CLI tooling at bin/cli/commands/compression.mjs provides runtime management without deployment cycles.
• Telemetry tables (compressionRunTelemetry, compressionAnalytics) and budget scripts (scripts/check/check-compression-budget.ts) ensure operational observability.

Frequently Asked Questions

What compression engines are available in OmniRoute?

OmniRoute ships with at least two built-in engines: caveman and stacked. These identifiers are registered in src/lib/db/migrations/102_compression_engines_map.sql and mapped to concrete transformer implementations. The engine registry design allows custom engines to be added without core code changes.

Does token compression add latency to LLM requests?

Compression introduces minimal overhead. The MCP Compression Worker pool handles transformations asynchronously for streaming requests, and decompression on the response path is optimized for speed. Per-request opt-in via the API key setting lets operators benchmark impact before global rollout.

How do I verify that compression is actually saving tokens?

Use the telemetry inspection command: omniroute compression stats --key <KEY>. This queries compressionRunTelemetry and compressionAnalytics to display tokens saved per request, engine utilization rates, and any decompression errors. For automated validation, integrate scripts/check/check-compression-budget.ts into CI pipelines.

Can I use different compression engines for different request types?

Yes. Since the compression object is specified per-request in the REST API body, clients can select engines dynamically based on content type, expected response length, or cost sensitivity. The API-key level default from migration 150_api_key_compression_enabled.sql serves as a fallback when no per-request override is provided.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}