How the NVIDIA NIM Integration Exposes Open-Weight Models for Council Use

The NVIDIA NIM integration exposes open-weight models by treating NIM as an OpenAI-compatible provider that auto-registers when NVIDIA_API_KEY is detected, routing requests to https://integrate.api.nvidia.com/v1 through a generic HTTP client that handles model selection via tiered slots.

The council-of-high-intelligence framework treats NVIDIA NIM as a first-class inference provider for its multi-agent coordination system. By leveraging the openai_compatible_api execution method, the NVIDIA NIM integration allows the council to access over 130 open-weight models—including DeepSeek, Qwen, and Nemotron—without custom adapters. This architecture enables seamless model switching and load balancing across the council's member seats.

Provider Auto-Detection via Environment Variables

The framework automatically detects NVIDIA NIM capabilities by scanning for the NVIDIA_API_KEY environment variable. In scripts/detect-providers.sh, the coordinator creates a provider entry named nvidia_nim with exec_method: "openai_compatible_api" when this key is present.

This detection mechanism ensures that sensitive credentials remain ephemeral—API keys are exposed only in-memory and never written to logs or configuration files.

Detection Logic Structure

The detection script emits a JSON structure defining the provider:

{
  "provider": "nvidia_nim",
  "exec_method": "openai_compatible_api",
  "base_url": "https://integrate.api.nvidia.com/v1",
  "api_key_env": "NVIDIA_API_KEY"
}

Configuring Open-Weight Model Slots

Open-weight models are exposed through the council's slot configuration system. The file configs/provider-model-slots.nim.example.yaml maps NVIDIA NIM's model catalog into tiered slots that the coordinator uses for member assignment.

Tiered Model Selection

The configuration supports tiered routing:

• High tier: Assigns the largest reasoning models (e.g., deepseek-ai/deepseek-v4-pro)
• Mid tier: Allocates smaller, faster variants (e.g., deepseek-ai/deepseek-v4-lite)

Example configuration:

# configs/provider-model-slots.nim.example.yaml

provider: nvidia_nim
models:
  - name: deepseek-ai/deepseek-v4-pro   # high‑tier model

    tier: high
  - name: deepseek-ai/deepseek-v4-lite  # mid‑tier model

    tier: mid
  # …add any of the 130+ open‑weight models NIM exposes…

OpenAI-Compatible Request Dispatch

All NIM seats use the openai_compatible_api execution method, enabling the council to treat NVIDIA's infrastructure identically to other OpenAI-compatible endpoints.

Authentication and Base URL

The dispatcher reads the NVIDIA_API_KEY from the environment and targets the base URL https://integrate.api.nvidia.com/v1. Requests are sent via HTTP POST to the /chat/completions endpoint, using standard OpenAI-compatible request schemas.

This path is shared with other providers such as Together, Fireworks, or vLLM, allowing the council to maintain a single HTTP client implementation while supporting diverse backends.

Provider Spread and Affinity Routing

The council's routing algorithm distributes members across available providers using spread and affinity rules defined in SKILL.md.

Provider Spread Algorithm

The algorithm treats nvidia_nim as a single provider regardless of how many models it hosts. Members are evenly distributed across all detected providers to prevent single-provider bottlenecks.

Council Member Affinity

Members that specify nvidia_nim in their council.provider_affinity configuration receive preferential assignment to NIM seats. Without explicit affinity, the coordinator assigns NIM seats only when other providers lack capacity.

Running the Council with NVIDIA NIM

Enable NIM integration by setting your API key and running the installation script:

export NVIDIA_API_KEY=nvapi-xxxxxx            # set your API key (never logged)

./install.sh                                   # registers the NIM provider

council-run --config configs/provider-model-slots.nim.example.yaml

The install.sh script invokes scripts/detect-providers.sh automatically, and council-run allocates council members according to the affinity and spread rules.

Summary

• Auto-detection: The scripts/detect-providers.sh script registers nvidia_nim when NVIDIA_API_KEY is present, setting exec_method: "openai_compatible_api".
• Model exposure: Open-weight models are configured via configs/provider-model-slots.nim.example.yaml using tiered slots (high/mid).
• Unified routing: Requests dispatch to https://integrate.api.nvidia.com/v1/chat/completions using the OpenAI-compatible HTTP client.
• Load balancing: Provider spread distributes members evenly, while provider_affinity allows preferential NIM assignment.

Frequently Asked Questions

How does the council authenticate with NVIDIA NIM?

The coordinator reads the NVIDIA_API_KEY environment variable at runtime and includes it in request headers. According to the source code in scripts/detect-providers.sh, this key is processed only in-memory and never persisted to logs or configuration files.

Can I use custom models not listed in the example configuration?

Yes. You can add any of the 130+ open-weight models available through NVIDIA NIM to your slot configuration file. The openai_compatible_api execution method supports any model identifier valid at the NIM endpoint.

How does the council handle failover if NVIDIA NIM is unavailable?

The routing algorithm's spread mechanism automatically distributes members across all available providers. If a NIM seat fails, the coordinator reassigns the member to seats from other providers (such as Together or Fireworks) that share the openai_compatible_api execution path.

What is the difference between high-tier and mid-tier model slots?

High-tier slots route to the largest reasoning models (e.g., DeepSeek-v4-pro) optimized for complex reasoning, while mid-tier slots use smaller, faster variants (e.g., DeepSeek-v4-lite) for latency-sensitive tasks. The coordinator selects slots based on council member requirements and current provider capacity.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}