# NVIDIA NIM Integration for Accessing Open-Weight Models in Council of High Intelligence

> Integrate NVIDIA NIM with Council of High Intelligence to easily access open-weight models. Discover seamless inference for DeepSeek, Kimi, and Qwen via the NVIDIA hosted API.

- Repository: [nyk/council-of-high-intelligence](https://github.com/0xNyk/council-of-high-intelligence)
- Tags: deep-dive
- Published: 2026-06-30

---

**The Council of High Intelligence (CoHI) treats NVIDIA NIM as an OpenAI-compatible provider, automatically detecting the `NVIDIA_API_KEY` environment variable to route requests to NVIDIA's hosted inference API for open-weight models like DeepSeek, Kimi, and Qwen.**

The Council of High Intelligence (CoHI) is an open-source framework for coordinating multiple AI models in a distributed "council" architecture. By implementing **NVIDIA NIM integration for accessing open-weight models**, CoHI enables seamless access to NVIDIA's catalog of hosted open-weight models—including DeepSeek, Kimi, MiniMax, GLM, and Qwen—through a standardized OpenAI-compatible endpoint, eliminating the need for local GPU infrastructure or separate provider accounts.

## Automatic Provider Detection

CoHI implements automatic discovery of NVIDIA NIM capabilities through environment variable inspection and shell-based detection scripts.

### Environment Variable Trigger

When the coordinator initializes, it scans for the presence of a `NVIDIA_API_KEY` environment variable. Upon detection, the system automatically registers a provider entry named **`nvidia_nim`** with the execution method `openai_compatible_api` in [`SKILL.md`](https://github.com/0xNyk/council-of-high-intelligence/blob/main/SKILL.md) lines 199‑202. This trigger requires no manual configuration files to enable the provider.

### Detection Script Reference

The shell script [`scripts/detect-providers.sh`](https://github.com/0xNyk/council-of-high-intelligence/blob/main/scripts/detect-providers.sh) contains explicit detection logic for NIM at line 90, marking it as a detectable OpenAI-compatible endpoint. The script checks for the `NVIDIA_API_KEY` variable and validates that the key conforms to the expected `nvapi-` prefix format issued by build.nvidia.com.

## Configuring NIM Seats

Seats utilizing the NIM provider are declared in YAML configuration files following the "provider-model-slots" schema. The reference implementation in [`configs/provider-model-slots.nim.example.yaml`](https://github.com/0xNyk/council-of-high-intelligence/blob/main/configs/provider-model-slots.nim.example.yaml) demonstrates the required parameters:

```yaml
seats:
  feynman:
    provider: nvidia_nim
    model: deepseek-ai/deepseek-v4-pro
    base_url: https://integrate.api.nvidia.com/v1
    api_key_env: NVIDIA_API_KEY
    reasoning_mode: mechanistic

```

### Required Configuration Parameters

Each NIM seat must specify four critical fields:

- **`provider`**: Must be set to `nvidia_nim` to trigger the NIM routing logic.
- **`model`**: The full model identifier (e.g., `deepseek-ai/deepseek-v4-pro`, `kimi-ai/kimi-v1`).
- **`base_url`**: Fixed to `https://integrate.api.nvidia.com/v1` for all NIM requests.
- **`api_key_env`**: References the environment variable name containing the API key (typically `NVIDIA_API_KEY`).

## Request Routing and Execution

CoHI dispatches NIM seat requests via HTTP rather than local sub-processes, utilizing the OpenAI-compatible API specification.

### HTTP Dispatch Mechanism

During council routing, seats configured with `exec_method: openai_compatible_api` (as defined in [`SKILL.gemini.md`](https://github.com/0xNyk/council-of-high-intelligence/blob/main/SKILL.gemini.md) lines 139‑148) are routed through HTTP POST requests to the specified `base_url`. The coordinator constructs the request header `Authorization: Bearer <key>` by reading the environment variable named in `api_key_env` at runtime.

### Security Architecture

The NVIDIA API key remains ephemeral throughout the execution lifecycle. According to the implementation in [`SKILL.md`](https://github.com/0xNyk/council-of-high-intelligence/blob/main/SKILL.md) line 202, the key is held only in memory during active request processing and is **never logged to transcripts, written to disk, or exposed in error traces**.

## Model Tiering and Selection

CoHI's auto-routing logic categorizes NIM models into performance tiers to optimize for capability versus latency.

### High-Tier Configuration

For complex reasoning tasks, the default **high** tier maps to `deepseek-ai/deepseek-v4-pro`, providing maximum parameter count and reasoning depth as specified in [`SKILL.md`](https://github.com/0xNyk/council-of-high-intelligence/blob/main/SKILL.md) lines 201‑202.

### Mid-Tier Alternatives

The **mid** tier utilizes smaller, faster variants from the same model families, enabling rapid iteration when full parameter inference is unnecessary. The coordinator automatically selects the appropriate tier based on the council's `--profile` flag and query complexity heuristics.

## Benefits of NIM Integration

Integrating NVIDIA NIM provides distinct architectural advantages for CoHI deployments:

- **Diverse Model Access**: Access open-weight models from DeepSeek, Kimi, MiniMax, GLM, Qwen, and Nemotron through a single credential set.
- **Reduced Infrastructure Overhead**: Eliminate local GPU requirements and model weight storage.
- **Bias Mitigation**: Mix NIM seats with other providers (OpenAI, Anthropic, Google) to increase response diversity and reduce shared-training-data bias as noted in [`configs/provider-model-slots.nim.example.yaml`](https://github.com/0xNyk/council-of-high-intelligence/blob/main/configs/provider-model-slots.nim.example.yaml) lines 14‑17.
- **Standardized Interface**: The OpenAI-compatible endpoint simplifies credential handling and request formatting across heterogeneous model sources.

## Practical Implementation Examples

### Exporting the API Key

Obtain a key from build.nvidia.com (keys start with `nvapi-`) and export it:

```bash
export NVIDIA_API_KEY=nvapi-XXXXXXXXXXXXXXXX

```

### Running a Council with NIM Configuration

Execute a council session using the example NIM configuration:

```bash
council --profile exploration-orthogonal \
        --models configs/provider-model-slots.nim.example.yaml \
        "What are the implications of quantum computing for AI alignment?"

```

### Verifying Available Models

Optionally verify accessible models before council execution:

```bash
curl -H "Authorization: Bearer $NVIDIA_API_KEY" \
     https://integrate.api.nvidia.com/v1/models | jq '.data[].id'

```

## Summary

- CoHI automatically detects NVIDIA NIM when the `NVIDIA_API_KEY` environment variable is present, registering the provider as `nvidia_nim` with `exec_method: openai_compatible_api`.
- Configuration occurs through YAML seat definitions in files like [`configs/provider-model-slots.nim.example.yaml`](https://github.com/0xNyk/council-of-high-intelligence/blob/main/configs/provider-model-slots.nim.example.yaml), specifying the model ID and `https://integrate.api.nvidia.com/v1` endpoint.
- Requests route via HTTP with ephemeral Bearer token authentication; keys are never persisted or logged.
- The integration supports model tiering (high/mid) and enables access to open-weight models without local infrastructure.

## Frequently Asked Questions

### How does CoHI authenticate requests to NVIDIA NIM?

The coordinator reads the API key from the environment variable specified in the seat's `api_key_env` field (typically `NVIDIA_API_KEY`) and injects it as an `Authorization: Bearer <token>` header. As implemented in [`SKILL.md`](https://github.com/0xNyk/council-of-high-intelligence/blob/main/SKILL.md), this key exists only in memory during the request lifecycle and is excluded from all logging and transcript generation.

### Which open-weight models are accessible through NIM integration?

The NVIDIA NIM integration provides access to models including DeepSeek-V4-Pro, Kimi-V1, MiniMax-Text-01, GLM-4, Qwen-2.5, and Nemotron-4. The specific model identifier must match the path format used in NVIDIA's model catalog (e.g., `deepseek-ai/deepseek-v4-pro`).

### How does routing differ between NIM seats and local agents?

NIM seats use `exec_method: openai_compatible_api`, which triggers HTTP dispatch to the `base_url` endpoint. Local agents instead execute as sub-processes or local API calls. This distinction is handled automatically by the coordinator based on the provider configuration in [`SKILL.gemini.md`](https://github.com/0xNyk/council-of-high-intelligence/blob/main/SKILL.gemini.md) lines 139‑148.

### Can I mix NVIDIA NIM seats with other providers in the same council?

Yes. CoHI explicitly supports heterogeneous councils mixing NIM seats with OpenAI, Anthropic, Google, and other providers. This configuration increases response diversity and mitigates shared-training-data bias, as documented in the example configuration comments.