How to Configure OmniRoute's Auto-Combo Engine for Self-Healing Routing

OmniRoute's Auto-Combo engine automatically creates virtual combos for auto/ prefixed requests, scores candidates using a 14-factor model, and applies circuit-breaker-aware self-healing to exclude unhealthy providers and fall back to safe modes.

The Auto-Combo engine in OmniRoute (release v3.8.50) enables intelligent, fault-tolerant routing without manual provider configuration. By leveraging real-time health monitoring, multi-factor scoring, and automatic recovery probes, it ensures high availability even when upstream providers degrade. This guide covers the architecture, configuration options, and self-healing mechanisms based on the source code in the diegosouzapw/OmniRoute repository.

Understanding the Auto-Combo Architecture

The Auto-Combo engine consists of six coordinated components that transform a simple auto/ model request into a routed, self-healing provider selection.

Core Components and Source Files

Component	Role	Source File
Prefix parser	Detects auto/ and extracts variants like auto/coding	open-sse/services/autoCombo/autoPrefix.ts
Virtual combo factory	Builds candidate pool from active connections	open-sse/services/autoCombo/virtualFactory.ts
Scoring engine	Computes 14-factor weighted scores	open-sse/services/autoCombo/scoring.ts
Self-healing module	Manages exclusions, circuit breakers, and probes	open-sse/services/autoCombo/selfHealing.ts
Router strategies	Implements selection algorithms	open-sse/services/autoCombo/engine.ts
Request handler	Entry point for chat requests	src/sse/handlers/chat.ts

Request Flow Through the Engine

When a client sends model: "auto/<variant>", the engine executes this pipeline:

1. handleChatCore() in src/sse/handlers/chat.ts detects the auto/ prefix
2. parseAutoPrefix() extracts the optional variant suffix
3. createVirtualAutoCombo() queries active connections and builds VirtualAutoComboCandidate objects
4. The self-healing module filters OPEN circuit-breakers and applies temporary exclusions
5. scorePool() with DEFAULT_WEIGHTS ranks remaining candidates
6. The configured router strategy selects the final provider
7. Normal combo execution dispatches the request

Configuring the Auto-Combo Scoring Engine

The scoring engine in open-sse/services/autoCombo/scoring.ts evaluates candidates using 9 core factors with 5 optional extensions. Understanding these weights lets you tune routing behavior for your workload.

Default Scoring Weights

The DEFAULT_WEIGHTS constant defines the baseline scoring model:

• Health status – Provider uptime and error rates
• Quota availability – Remaining rate limits and token budgets
• Cost efficiency – Price per token for the requested model tier
• Latency percentile – Historical response time at P95/P99
• Task-fit score – Model capability match for the variant (e.g., coding, vision)
• Provider tier – Premium vs. standard classification
• Geographic proximity – Regional latency optimization
• Recent success rate – Weighted average of last N requests
• Circuit-breaker confidence – Penalty for recent failures

Optional factors include embedding latency, streaming support, and custom metadata tags.

Customizing Score Weights

While the default weights suit most deployments, you can influence scoring through variant-specific configuration. The virtual factory reads provider metadata and connection settings to adjust factor weights dynamically.

For example, a auto/coding variant might prioritize:

• Task-fit score (increased weight for code-capable models)
• Latency percentile (reduced weight for batch processing tolerance)
• Cost efficiency (balanced against quality requirements)

Enabling and Tuning Self-Healing Behavior

The self-healing module in open-sse/services/autoCombo/selfHealing.ts provides three interconnected mechanisms for automatic failure recovery.

Circuit-Breaker Integration

The engine respects provider circuit-breaker states from the broader OmniRoute health system:

• CLOSED – Provider eligible for selection
• HALF_OPEN – Available with reduced confidence scoring
• OPEN – Automatically excluded from candidate pool

Temporary Exclusion with Cooldown Probing

When a provider fails during request execution, the self-healing module:

1. Applies a temporary exclusion with exponential backoff
2. Schedules asynchronous probe requests to test recovery
3. Reintegrates the provider upon successful probe completion

The probe logic uses lightweight health checks rather than full inference requests to minimize overhead.

Incident Mode Activation

When more than 50% of candidates are in OPEN state, the engine switches to incident mode:

• Reduces concurrency limits across all providers
• Increases probe frequency for faster recovery detection
• May activate fallback strategies depending on configuration
• Logs incident events for operational visibility

Configuring Router Strategies

The router strategy determines how scored candidates become final selections. OmniRoute provides 19 routing strategies defined in src/shared/constants/routingStrategies.ts.

Available Strategy Values

The ROUTING_STRATEGY_VALUES enum includes:

• rules – Policy-based selection using configured constraints (default for Auto-Combo)
• cost – Lowest estimated cost
• latency – Lowest predicted latency
• sla-aware – Balances latency against SLA commitments
• lkgp – Last-known-good provider with fallback
• auto – Delegates to Auto-Combo internal logic

Strategy Selection Methods

You can configure strategies at two levels:

Per-request override:

{
  "model": "auto/coding",
  "routing": {
    "strategy": "latency"
  }
}

Per-combo default in configuration:

The virtual factory applies strategy selection during createVirtualAutoCombo() execution, merging request-level overrides with system defaults.

Practical Configuration Examples

Basic Auto-Combo Invocation

Send a request with the auto/ prefix to trigger the engine:

curl -X POST http://omniroute/api/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "auto/general",
    "messages": [{"role": "user", "content": "Hello"}]
  }'

The parseAutoPrefix() function extracts "general" as the variant, which influences task-fit scoring.

Variant-Specific Routing

Target specialized capabilities through variant naming:

Variant	Typical Use Case	Scoring Emphasis
auto/coding	Software development	Code models, lower latency weight
auto/vision	Image analysis	Multimodal capability, context window
auto/fast	Real-time applications	Aggressive latency optimization
auto/cheap	Batch processing	Cost minimization, relaxed latency

Self-Healing Monitoring

Monitor self-healing activity through built-in metrics. The module tracks:

• Exclusion count and duration per provider
• Probe success/failure rates
• Incident mode transitions
• Effective pool size over time

Integrating Auto-Combo with Existing Combos

The Auto-Combo engine seamlessly integrates with OmniRoute's standard combo pipeline. After candidate selection, requests flow through:

1. The same authentication and credential resolution as static combos
2. Identical request transformation and prompt formatting
3. Shared response streaming and error handling
4. Unified logging and observability

This design ensures that Auto-Combo benefits from all existing combo features without separate code paths.

Summary

• Auto-Combo triggers on auto/ prefixes, parsed by parseAutoPrefix() in open-sse/services/autoCombo/autoPrefix.ts
• Virtual candidate pools are built dynamically by createVirtualAutoCombo() from active provider connections
• 14-factor scoring with 9 core weights in DEFAULT_WEIGHTS ranks candidates by health, cost, latency, and fit
• Self-healing automatically excludes OPEN circuit-breakers, probes recovering providers, and enters incident mode when >50% of candidates fail
• 19 router strategies including rules, cost, latency, and sla-aware control final selection
• Zero static configuration required—providers are discovered from live connections and credentials

Frequently Asked Questions

What triggers the Auto-Combo engine to activate?

Any request with a model name beginning with auto/ activates the engine. The handleChatCore() function in src/sse/handlers/chat.ts detects this prefix and invokes the virtual factory. The optional variant suffix (e.g., auto/coding) influences scoring weights but does not change the activation logic.

How does the self-healing mechanism know when to exclude a provider?

The self-healing module in open-sse/services/autoCombo/selfHealing.ts monitors circuit-breaker states and recent failure patterns. Providers with OPEN breakers are immediately excluded. Additional temporary exclusions apply after request failures, with automatic probe requests testing recovery before reintegration.

Can I use custom scoring weights for specific workloads?

While the DEFAULT_WEIGHTS in open-sse/services/autoCombo/scoring.ts provide system-wide defaults, variant prefixes influence weight adjustments. The virtual factory passes variant metadata to the scoring engine, which applies modifier logic. For advanced customization, fork the scoring module and adjust the 14-factor calculation.

What happens when all providers are unhealthy?

When more than 50% of candidates are excluded, the engine enters incident mode: it reduces concurrency, increases probe frequency, and may degrade to safer strategies. If no candidates remain after filtering, the request fails fast with a clear error indicating pool exhaustion, enabling upstream retries or queue-based backoff.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}