How Multi-Provider Auto-Routing Works in the Council of High Intelligence

The Council of High Intelligence distributes its 18 agent members across multiple LLM providers through a three-stage deterministic process involving environment detection, constraint-based routing algorithms, and tiered model assignment.

The Council of High Intelligence (CoHI) is an open-source multi-agent deliberation framework that automatically balances cognitive workloads across disparate large language model providers. This multi-provider auto-routing capability eliminates manual configuration while ensuring deterministic, reproducible seat assignments based on provider availability, agent compatibility, and deliberate architectural constraints. The system is governed by the coordinator specification in SKILL.md and supporting configuration assets.

Provider Detection Pipeline

When executing without explicit --models mappings or the --no-auto-route flag, the coordinator initiates environment discovery through scripts/detect-providers.sh.

Environment Scanning with detect-providers.sh

The detection script probes the host runtime for six provider categories, each with distinct discovery heuristics:

• anthropic: Always available (host runtime)
• openai: Detected via codex binary presence; executes via codex_exec
• google: Detected via gemini binary presence; executes via gemini_cli
• ollama: Requires ollama binary plus running server; uses ollama_run
• cursor_cli: Detected via cursor-agent binary presence
• nvidia_nim: Requires NVIDIA_API_KEY environment variable and reachable endpoint; uses openai_compatible_api

The script outputs a JSON object containing a providers array, provider count, and a multi_provider boolean flag. If only one provider is detected (typically Anthropic), the coordinator skips auto-routing and assigns every member its front-matter default model.

Detection Output Structure

The JSON output from detect-providers.sh feeds directly into the routing algorithm, providing:

• Available provider list with execution methods
• Model availability per provider (e.g., opus, sonnet, haiku for Anthropic; first 5 local models for Ollama)
• API endpoint configurations for OpenAI-compatible providers

The Auto-Routing Algorithm

Defined in SKILL.md (lines 191–204), the algorithm executes after detection and before deliberation begins, processing hard constraints before soft preferences.

Hard Constraints: Polarity and Spread

The algorithm enforces two non-negotiable architectural requirements:

1. Polarity-pair separation: Members listed as polarity pairs in their front-matter must occupy different providers to ensure adversarial diversity.
2. Provider spread: Members distribute evenly across all detected providers using ⌊M/N⌋ or ⌈M/N⌉ calculations, where M is member count and N is provider count. Aggregator providers (nvidia_nim and cursor_cli) count as a single provider entity for spreading calculations.

Soft Constraints: Affinity and Tier Matching

After satisfying hard constraints, the algorithm optimizes assignments using:

• Provider affinity: Members declaring council.provider_affinity in front-matter receive preferred providers when capacity permits, processed in declared order.
• Tier matching: Model tiers (high vs. mid) are determined from configs/auto-route-defaults.yaml. For example, Anthropic opus maps to high tier, sonnet to mid tier. NVIDIA NIM high tier defaults to deepseek-ai/deepseek-v4-pro, while mid tier uses deepseek-ai/deepseek-v4-flash.

OpenAI-Compatible Seat Hydration

For providers utilizing the openai_compatible_api execution method (such as NVIDIA NIM), the coordinator performs runtime hydration by reading base_url and the environment variable specified in api_key_env from the detection JSON. The API key remains in memory only and never persists to disk.

Chairman Selection Hierarchy

Following seat assignment (Step 1.7 in SKILL.md), the coordinator selects a Chairman—the synthesis model producing the final verdict—using a three-stage hierarchy:

1. Explicit override: Command-line argument --chairman <name> takes absolute precedence.
2. Configuration override: A non-null chairman: block in auto-route-defaults.yaml overrides automatic selection.
3. Auto-select: The algorithm chooses the highest-tier model among detected providers, preferring a provider not already hosting panel members to avoid inference concentration. The chairman_defaults: section in auto-route-defaults.yaml defines tier mappings for this selection.

When operating in single-provider mode, the Chairman simply reuses that provider's highest-tier model (typically Anthropic opus).

Inspecting the Routing Table

Users can preview assignments without executing deliberation using the --dry-route flag:

# Standard auto-routing execution

council run "Explain quantum-safe encryption trade-offs"

# Preview routing table only

council run --dry-route "Explain quantum-safe encryption trade-offs"

The --dry-route output displays the complete mapping:

feynman      → anthropic   → opus           → subagent
torvalds     → openai      → gpt-5.4        → codex_exec
karpathy     → cursor_cli  → gpt-5.4-high   → cursor_cli
...

This table reveals member-to-provider assignments, concrete model selections (high/mid tier), and execution methods before API calls initiate.

Summary

• Multi-provider auto-routing activates automatically when detect-providers.sh finds multiple available providers and the user omits --no-auto-route.
• The algorithm enforces hard constraints (polarity separation, even provider spread) before applying soft preferences (provider affinity, tier matching).
• Configuration lives in configs/auto-route-defaults.yaml for tier mappings and chairman defaults, while SKILL.md contains the routing logic specification.
• OpenAI-compatible providers receive dynamic credential hydration without persistent storage of API keys.
• Use --dry-route to validate assignments before consuming inference credits.

Frequently Asked Questions

What triggers multi-provider auto-routing versus single-provider mode?

Auto-routing activates when scripts/detect-providers.sh identifies two or more available providers and the user has not specified --models manual mappings or --no-auto-route. If only Anthropic (or any single provider) is detected, the coordinator bypasses routing and assigns all 18 members their default front-matter models.

How does the Council handle API keys for providers like NVIDIA NIM?

The coordinator reads API keys from environment variables specified in api_key_env within the detection JSON output. These credentials exist only in runtime memory during OpenAI-compatible seat hydration and are never written to configuration files or logs, maintaining security while enabling multi-provider execution.

Can I force specific agents to use particular providers?

Yes. Agents can declare council.provider_affinity in their front-matter metadata as an ordered list of preferred providers. While the algorithm prioritizes hard constraints (polarity separation, even distribution) first, it respects affinity preferences when capacity allows, assigning agents to their earliest listed available provider.

What happens if the Chairman model fails or becomes unavailable?

The Chairman selection hierarchy defined in SKILL.md provides multiple fallback levels. If the auto-selected highest-tier model is unreachable, the coordinator re-evaluates available providers from the detection JSON and selects the next highest-tier candidate from chairman_defaults in auto-route-defaults.yaml, ensuring deliberation continuity without manual intervention.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}