Testing Best Practices for OmniRoute Applications: A Layered Testing Strategy

OmniRoute enforces a rigorous three-tier testing pyramid—unit tests, Vitest-based MCP/combo tests, and optional live integration suites—mandating a minimum 60% coverage gate across statements, lines, functions, and branches.

OmniRoute is an open-source routing engine supporting 341 LLM providers with complex combo strategies. Implementing testing best practices for OmniRoute applications ensures that new providers, routing combinations, and security guardrails integrate safely without regressions. The repository defines these standards in AGENTS.md, AUTO-COMBO.md, and GUARDRAILS.md, with automated enforcement via CI quality gates.

Execute the Three-Tier Test Suite

OmniRoute separates concerns across three distinct test scopes to balance speed and thoroughness. According to the Testing & Coverage section in docs/routing/AUTO-COMBO.md, you must run each tier for comprehensive validation.

• Unit tests use the native Node test runner for fast, hermetic checks without external services: npm run test:unit
• Vitest tests cover MCP tools, auto-combo routing logic, and cache behavior: npm run test:vitest
• Coverage gate validates the 60% floor across all metrics: npm run test:coverage

Integration and E2E tests operate only when explicitly enabled via RUN_COMBO_LIVE=1 or NEXT_PUBLIC_OMNIROUTE_E2E_MODE=true, ensuring the default suite remains fast and isolated.

Maintain the 60% Global Coverage Floor

The CI pipeline fails if any of four coverage metrics—statements, lines, functions, or branches—drops below 60%. As documented in the Coverage gate section of AGENTS.md, the npm run test:coverage command enforces this threshold automatically.

For critical modules such as routing, guardrails, and database layers, docs/architecture/QUALITY_GATES.md mandates per-module coverage floors enforced by npm run quality:ratchet. When you modify these modules, add or update tests to maintain the module-specific score above its ratchet threshold.

Write Regression-Guard Tests for Security

Every new guardrail, public-credential path, or error-sanitization rule requires a corresponding regression test. The Testing section of docs/security/GUARDRAILS.md explicitly requires a test for each new rule in the tests/unit/ directory.

For example, when adding guardrail logic, create a test alongside existing suites like tests/unit/publicCreds.test.ts or tests/unit/error-message-sanitization.test.ts:

// tests/unit/guardrails/new-guardrail.test.ts
import { enforceMyGuardrail } from '@/src/lib/guardrails/myGuardrail';
import { expect, test } from 'vitest';

test('rejects disallowed token pattern', () => {
  const badPrompt = 'DROP TABLE users; --';
  const result = enforceMyGuardrail(badPrompt);
  expect(result.allowed).toBe(false);
  expect(result.reason).toContain('dangerous token');
});

This practice ensures that security-critical paths remain protected against future refactors.

Keep Tests Hermetic with Isolated Environments

OmniRoute tests must not rely on external network calls unless explicitly enabled. The CI-Testing section of docs/reference/ENVIRONMENT.md specifies test-only variables that enforce isolation:

• DATA_DIR=/tmp/omniroute-test — Isolates test data from production
• OMNIROUTE_DISABLE_BACKGROUND_SERVICES=1 — Disables background workers
• OMNIROUTE_SKIP_DB_HEALTHCHECK=1 — Bypasses health checks for unit tests

Never use production secrets in tests. Instead, use stubbed provider configs from the tests/fixtures/ directory to mock external dependencies.

Validate Routing Logic with Combo Tests

OmniRoute supports 19 distinct combo routing strategies. Validate changes using the deterministic test matrix that exercises routing decisions without live credentials:

# Verify all 19 strategies deterministically

npm run test:combo:matrix

For real-upstream validation, run npm run test:combo:live with valid credentials and RUN_COMBO_LIVE=1 set. This tier confirms that provider integrations respond correctly to the auto-combo logic documented in docs/routing/AUTO-COMBO.md.

Enforce Advanced Quality Gates

Beyond coverage, OmniRoute employs mutation testing via nightly Stryker runs to detect weak test assertions. As specified in docs/ops/QUALITY_GATE_PLAYBOOK.md, maintain the project-specific mutation score threshold to ensure tests exercise error paths, not just happy paths.

Before submitting a PR, run the local CI validation to ensure gates pass:

npm run check        # Lint + test

npm run test:combo:matrix  # Deterministic routing validation

Summary

• Run three test tiers: Unit (Node native), Vitest (MCP/combo), and integration (opt-in live) to cover all code paths.
• Enforce 60% coverage: Global minimum for statements, lines, functions, and branches; respect per-module ratchets for critical systems.
• Add regression guards: Every security feature in docs/security/GUARDRAILS.md requires a matching unit test in tests/unit/.
• Isolate environments: Use DATA_DIR and OMNIROUTE_DISABLE_BACKGROUND_SERVICES variables to keep tests hermetic.
• Validate routing: Use npm run test:combo:matrix to check all 19 combo strategies deterministically.

Frequently Asked Questions

What is the minimum code coverage required for OmniRoute applications?

OmniRoute enforces a 60% minimum across four metrics: statements, lines, functions, and branches. The npm run test:coverage command fails the build if any metric falls below this threshold. Additionally, critical modules such as routing and guardrails maintain higher per-module floors via the quality:ratchet command defined in docs/architecture/QUALITY_GATES.md.

How do I run tests without hitting live LLM providers?

Run npm run test:unit and npm run test:vitest for hermetic suites that use fixtures from tests/fixtures/ instead of live APIs. Use npm run test:combo:matrix to test the 19 routing strategies deterministically. Only touch live providers when you explicitly set RUN_COMBO_LIVE=1 or NEXT_PUBLIC_OMNIROUTE_E2E_MODE=true before running integration commands.

Where should I place new tests when adding security guardrails?

Place unit tests in the tests/unit/ directory following the convention of existing files like tests/unit/publicCreds.test.ts and tests/unit/error-message-sanitization.test.ts. The Testing section of docs/security/GUARDRAILS.md mandates that every new guardrail, credential rule, or error-sanitization path include a corresponding regression test in this folder.

What environment variables ensure hermetic testing?

Set DATA_DIR=/tmp/omniroute-test to isolate data, OMNIROUTE_DISABLE_BACKGROUND_SERVICES=1 to stop background workers, and OMNIROUTE_SKIP_DB_HEALTHCHECK=1 to bypass health checks during unit tests. These variables are documented in the CI-Testing section of docs/reference/ENVIRONMENT.md and prevent tests from bleeding into production systems.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}