Performance Benchmarks for OmniRoute: Technical Reference and Measurement Guide

OmniRoute does not ship conventional latency‑versus‑throughput benchmarks for the full routing stack; instead, it provides deterministic A/B benchmarks for compression engines and a comprehensive suite of over 21,000 unit and integration tests that act as performance sanity checks.

The OmniRoute repository focuses its performance validation on deterministic, reproducible measurements of its core cost‑saving components rather than traditional end‑to‑end load testing. This approach ensures that token compression efficiency and routing logic overhead remain predictable across continuous integration runs. Developers looking for system‑level metrics such as requests‑per‑second or average round‑trip time must instrument custom load‑testing scripts against the deployed /v1 endpoints.

Compression Engine A/B Benchmarks

OmniRoute’s primary performance measurement tool is the deterministic benchmark harness located in open-sse/services/compression/harness/benchmark.ts. This file executes A/B tests for every built‑in compression engine—including lite, caveman, RTK, aggressive, and ultra—against a controlled corpus of realistic workloads.

Benchmark Metrics and Corpus

The benchmark calculates three critical metrics for each engine:

• meanSavingsPercent – The average percentage of tokens saved across the corpus.
• meanRetention – A quality score indicating how much semantic content is preserved after compression.
• totalCompressedTokens – The absolute count of tokens remaining after compression.

The benchmark uses a static BENCHMARK_CORPUS divided into three task groups that mirror typical OmniRoute workloads: prose, tool‑output, and JSON. Because the harness avoids non‑deterministic functions such as Date.now() or Math.random(), results remain identical across CI runs, enabling reliable regression detection.

Running the Compression Benchmarks

You can execute the compression benchmarks programmatically to evaluate engine performance in your environment:

// Example: Run the built‑in compression benchmark programmatically
import { benchmarkEngines } from '@/open-sse/services/compression/harness/benchmark.ts';

(async () => {
  const results = await benchmarkEngines();
  console.table(results.summary); // shows meanSavingsPercent, meanRetention, etc.
})();

The core execution logic resides in open-sse/services/compression/harness/runner.ts, which iterates the corpus through each registered engine and aggregates the statistical results.

Unit and Integration Test Performance Validation

Beyond explicit benchmarks, OmniRoute leverages its extensive test suite—comprising over 21,000 unit and integration tests—to enforce performance boundaries. These tests validate that critical paths complete within expected time constraints.

Running the full validation suite ensures that routing logic, MCP tool execution, and A2A server responses stay within millisecond‑scale latency bounds:

# Run the full test suite (includes performance‑related sanity checks)

npm run test:all

Key areas covered by this indirect performance validation include:

• Combo routing logic in open-sse/services/combo.ts, verified by tests such as tests/unit/wildcard-router.test.ts.
• Fallback timing and error‑recovery paths in the routing layer.
• Token‑budget gating enforcement to prevent budget overruns in src/app/api/v1/**/route.ts (e.g., chat/completions/route.ts).

Key Files and Implementation Details

The following source files constitute OmniRoute’s complete performance measurement infrastructure:

File	Role in Performance Measurement
open-sse/services/compression/harness/benchmark.ts	Deterministic A/B benchmark for every compression engine, reporting token‑savings and retention.
open-sse/services/compression/harness/runner.ts	Core harness that executes the benchmark corpus against a given engine.
tests/**/*.test.ts	> 21 000 unit/integration tests that verify routing latency, fallback timing, and budget‑gate enforcement.
src/app/api/v1/**/route.ts	Entry point whose execution path is exercised by the test suite to ensure sub‑millisecond overhead.
open-sse/services/combo.ts	Combo routing logic whose performance is indirectly validated by combo‑related tests.

Summary

• OmniRoute prioritizes compression engine benchmarks over traditional throughput metrics, measuring meanSavingsPercent and meanRetention via deterministic A/B tests.
• The benchmark harness in open-sse/services/compression/harness/benchmark.ts uses a static corpus of prose, tool‑output, and JSON to ensure reproducible results.
• Over 21,000 unit and integration tests act as performance sanity checks, validating that routing and compression logic executes within millisecond bounds.
• No official end‑to‑end latency benchmarks are published; system‑level throughput testing requires custom scripts targeting the /v1 API endpoints.

Frequently Asked Questions

Does OmniRoute publish official latency or throughput benchmarks?

No. According to the OmniRoute source code, the repository does not publish end‑to‑end latency numbers such as requests per second or average round‑trip time. Performance validation focuses on deterministic compression metrics and test suite execution times rather than network‑level throughput.

How do I measure end‑to‑end routing performance?

You must create custom load‑testing scripts that invoke the /v1 endpoints (e.g., /v1/chat/completions) and measure observed response times under your specific concurrency and hardware constraints. The existing test suite validates functional correctness and basic latency bounds but does not simulate production load scenarios.

Which compression engines are included in the A/B benchmarks?

The benchmark in open-sse/services/compression/harness/benchmark.ts tests all built‑in engines, including lite, caveman, RTK, aggressive, and ultra. Each engine is evaluated against the same BENCHMARK_CORPUS to calculate token savings and retention scores.

Are the compression benchmarks deterministic?

Yes. The benchmark harness explicitly avoids non‑deterministic sources such as Date.now() or Math.random(), ensuring that every CI run produces identical results for the same code version. This design allows developers to detect performance regressions in compression efficiency with absolute confidence.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}