# How the NVIDIA NIM Integration Exposes Open-Weight Models for Council Use

> Learn how NVIDIA NIM integrates open-weight models for Council use by acting as an OpenAI-compatible provider routed through a generic HTTP client.

- Repository: [nyk/council-of-high-intelligence](https://github.com/0xNyk/council-of-high-intelligence)
- Tags: how-to-guide
- Published: 2026-07-03

---

**The NVIDIA NIM integration exposes open-weight models by treating NIM as an OpenAI-compatible provider that auto-registers when `NVIDIA_API_KEY` is detected, routing requests to `https://integrate.api.nvidia.com/v1` through a generic HTTP client that handles model selection via tiered slots.**

The council-of-high-intelligence framework treats NVIDIA NIM as a first-class inference provider for its multi-agent coordination system. By leveraging the `openai_compatible_api` execution method, the NVIDIA NIM integration allows the council to access over 130 open-weight models—including DeepSeek, Qwen, and Nemotron—without custom adapters. This architecture enables seamless model switching and load balancing across the council's member seats.

## Provider Auto-Detection via Environment Variables

The framework automatically detects NVIDIA NIM capabilities by scanning for the `NVIDIA_API_KEY` environment variable. In [`scripts/detect-providers.sh`](https://github.com/0xNyk/council-of-high-intelligence/blob/main/scripts/detect-providers.sh), the coordinator creates a provider entry named `nvidia_nim` with `exec_method: "openai_compatible_api"` when this key is present.

This detection mechanism ensures that sensitive credentials remain ephemeral—API keys are exposed only in-memory and never written to logs or configuration files.

### Detection Logic Structure

The detection script emits a JSON structure defining the provider:

```json
{
  "provider": "nvidia_nim",
  "exec_method": "openai_compatible_api",
  "base_url": "https://integrate.api.nvidia.com/v1",
  "api_key_env": "NVIDIA_API_KEY"
}

```

## Configuring Open-Weight Model Slots

Open-weight models are exposed through the council's slot configuration system. The file [`configs/provider-model-slots.nim.example.yaml`](https://github.com/0xNyk/council-of-high-intelligence/blob/main/configs/provider-model-slots.nim.example.yaml) maps NVIDIA NIM's model catalog into tiered slots that the coordinator uses for member assignment.

### Tiered Model Selection

The configuration supports tiered routing:

- **High tier**: Assigns the largest reasoning models (e.g., `deepseek-ai/deepseek-v4-pro`)
- **Mid tier**: Allocates smaller, faster variants (e.g., `deepseek-ai/deepseek-v4-lite`)

Example configuration:

```yaml

# configs/provider-model-slots.nim.example.yaml

provider: nvidia_nim
models:
  - name: deepseek-ai/deepseek-v4-pro   # high‑tier model

    tier: high
  - name: deepseek-ai/deepseek-v4-lite  # mid‑tier model

    tier: mid
  # …add any of the 130+ open‑weight models NIM exposes…

```

## OpenAI-Compatible Request Dispatch

All NIM seats use the `openai_compatible_api` execution method, enabling the council to treat NVIDIA's infrastructure identically to other OpenAI-compatible endpoints.

### Authentication and Base URL

The dispatcher reads the `NVIDIA_API_KEY` from the environment and targets the base URL `https://integrate.api.nvidia.com/v1`. Requests are sent via HTTP POST to the `/chat/completions` endpoint, using standard OpenAI-compatible request schemas.

This path is shared with other providers such as Together, Fireworks, or vLLM, allowing the council to maintain a single HTTP client implementation while supporting diverse backends.

## Provider Spread and Affinity Routing

The council's routing algorithm distributes members across available providers using spread and affinity rules defined in [`SKILL.md`](https://github.com/0xNyk/council-of-high-intelligence/blob/main/SKILL.md).

### Provider Spread Algorithm

The algorithm treats `nvidia_nim` as a single provider regardless of how many models it hosts. Members are evenly distributed across all detected providers to prevent single-provider bottlenecks.

### Council Member Affinity

Members that specify `nvidia_nim` in their `council.provider_affinity` configuration receive preferential assignment to NIM seats. Without explicit affinity, the coordinator assigns NIM seats only when other providers lack capacity.

## Running the Council with NVIDIA NIM

Enable NIM integration by setting your API key and running the installation script:

```bash
export NVIDIA_API_KEY=nvapi-xxxxxx            # set your API key (never logged)

./install.sh                                   # registers the NIM provider

council-run --config configs/provider-model-slots.nim.example.yaml

```

The [`install.sh`](https://github.com/0xNyk/council-of-high-intelligence/blob/main/install.sh) script invokes [`scripts/detect-providers.sh`](https://github.com/0xNyk/council-of-high-intelligence/blob/main/scripts/detect-providers.sh) automatically, and `council-run` allocates council members according to the affinity and spread rules.

## Summary

- **Auto-detection**: The [`scripts/detect-providers.sh`](https://github.com/0xNyk/council-of-high-intelligence/blob/main/scripts/detect-providers.sh) script registers `nvidia_nim` when `NVIDIA_API_KEY` is present, setting `exec_method: "openai_compatible_api"`.
- **Model exposure**: Open-weight models are configured via [`configs/provider-model-slots.nim.example.yaml`](https://github.com/0xNyk/council-of-high-intelligence/blob/main/configs/provider-model-slots.nim.example.yaml) using tiered slots (high/mid).
- **Unified routing**: Requests dispatch to `https://integrate.api.nvidia.com/v1/chat/completions` using the OpenAI-compatible HTTP client.
- **Load balancing**: Provider spread distributes members evenly, while `provider_affinity` allows preferential NIM assignment.

## Frequently Asked Questions

### How does the council authenticate with NVIDIA NIM?

The coordinator reads the `NVIDIA_API_KEY` environment variable at runtime and includes it in request headers. According to the source code in [`scripts/detect-providers.sh`](https://github.com/0xNyk/council-of-high-intelligence/blob/main/scripts/detect-providers.sh), this key is processed only in-memory and never persisted to logs or configuration files.

### Can I use custom models not listed in the example configuration?

Yes. You can add any of the 130+ open-weight models available through NVIDIA NIM to your slot configuration file. The `openai_compatible_api` execution method supports any model identifier valid at the NIM endpoint.

### How does the council handle failover if NVIDIA NIM is unavailable?

The routing algorithm's spread mechanism automatically distributes members across all available providers. If a NIM seat fails, the coordinator reassigns the member to seats from other providers (such as Together or Fireworks) that share the `openai_compatible_api` execution path.

### What is the difference between high-tier and mid-tier model slots?

High-tier slots route to the largest reasoning models (e.g., DeepSeek-v4-pro) optimized for complex reasoning, while mid-tier slots use smaller, faster variants (e.g., DeepSeek-v4-lite) for latency-sensitive tasks. The coordinator selects slots based on council member requirements and current provider capacity.