NVIDIA NIM Integration for Accessing Open-Weight Models in Council of High Intelligence

The Council of High Intelligence (CoHI) treats NVIDIA NIM as an OpenAI-compatible provider, automatically detecting the NVIDIA_API_KEY environment variable to route requests to NVIDIA's hosted inference API for open-weight models like DeepSeek, Kimi, and Qwen.

The Council of High Intelligence (CoHI) is an open-source framework for coordinating multiple AI models in a distributed "council" architecture. By implementing NVIDIA NIM integration for accessing open-weight models, CoHI enables seamless access to NVIDIA's catalog of hosted open-weight models—including DeepSeek, Kimi, MiniMax, GLM, and Qwen—through a standardized OpenAI-compatible endpoint, eliminating the need for local GPU infrastructure or separate provider accounts.

Automatic Provider Detection

CoHI implements automatic discovery of NVIDIA NIM capabilities through environment variable inspection and shell-based detection scripts.

Environment Variable Trigger

When the coordinator initializes, it scans for the presence of a NVIDIA_API_KEY environment variable. Upon detection, the system automatically registers a provider entry named nvidia_nim with the execution method openai_compatible_api in SKILL.md lines 199‑202. This trigger requires no manual configuration files to enable the provider.

Detection Script Reference

The shell script scripts/detect-providers.sh contains explicit detection logic for NIM at line 90, marking it as a detectable OpenAI-compatible endpoint. The script checks for the NVIDIA_API_KEY variable and validates that the key conforms to the expected nvapi- prefix format issued by build.nvidia.com.

Configuring NIM Seats

Seats utilizing the NIM provider are declared in YAML configuration files following the "provider-model-slots" schema. The reference implementation in configs/provider-model-slots.nim.example.yaml demonstrates the required parameters:

seats:
  feynman:
    provider: nvidia_nim
    model: deepseek-ai/deepseek-v4-pro
    base_url: https://integrate.api.nvidia.com/v1
    api_key_env: NVIDIA_API_KEY
    reasoning_mode: mechanistic

Required Configuration Parameters

Each NIM seat must specify four critical fields:

• provider: Must be set to nvidia_nim to trigger the NIM routing logic.
• model: The full model identifier (e.g., deepseek-ai/deepseek-v4-pro, kimi-ai/kimi-v1).
• base_url: Fixed to https://integrate.api.nvidia.com/v1 for all NIM requests.
• api_key_env: References the environment variable name containing the API key (typically NVIDIA_API_KEY).

Request Routing and Execution

CoHI dispatches NIM seat requests via HTTP rather than local sub-processes, utilizing the OpenAI-compatible API specification.

HTTP Dispatch Mechanism

During council routing, seats configured with exec_method: openai_compatible_api (as defined in SKILL.gemini.md lines 139‑148) are routed through HTTP POST requests to the specified base_url. The coordinator constructs the request header Authorization: Bearer <key> by reading the environment variable named in api_key_env at runtime.

Security Architecture

The NVIDIA API key remains ephemeral throughout the execution lifecycle. According to the implementation in SKILL.md line 202, the key is held only in memory during active request processing and is never logged to transcripts, written to disk, or exposed in error traces.

Model Tiering and Selection

CoHI's auto-routing logic categorizes NIM models into performance tiers to optimize for capability versus latency.

High-Tier Configuration

For complex reasoning tasks, the default high tier maps to deepseek-ai/deepseek-v4-pro, providing maximum parameter count and reasoning depth as specified in SKILL.md lines 201‑202.

Mid-Tier Alternatives

The mid tier utilizes smaller, faster variants from the same model families, enabling rapid iteration when full parameter inference is unnecessary. The coordinator automatically selects the appropriate tier based on the council's --profile flag and query complexity heuristics.

Benefits of NIM Integration

Integrating NVIDIA NIM provides distinct architectural advantages for CoHI deployments:

• Diverse Model Access: Access open-weight models from DeepSeek, Kimi, MiniMax, GLM, Qwen, and Nemotron through a single credential set.
• Reduced Infrastructure Overhead: Eliminate local GPU requirements and model weight storage.
• Bias Mitigation: Mix NIM seats with other providers (OpenAI, Anthropic, Google) to increase response diversity and reduce shared-training-data bias as noted in configs/provider-model-slots.nim.example.yaml lines 14‑17.
• Standardized Interface: The OpenAI-compatible endpoint simplifies credential handling and request formatting across heterogeneous model sources.

Practical Implementation Examples

Exporting the API Key

Obtain a key from build.nvidia.com (keys start with nvapi-) and export it:

export NVIDIA_API_KEY=nvapi-XXXXXXXXXXXXXXXX

Running a Council with NIM Configuration

Execute a council session using the example NIM configuration:

council --profile exploration-orthogonal \
        --models configs/provider-model-slots.nim.example.yaml \
        "What are the implications of quantum computing for AI alignment?"

Verifying Available Models

Optionally verify accessible models before council execution:

curl -H "Authorization: Bearer $NVIDIA_API_KEY" \
     https://integrate.api.nvidia.com/v1/models | jq '.data[].id'

Summary

• CoHI automatically detects NVIDIA NIM when the NVIDIA_API_KEY environment variable is present, registering the provider as nvidia_nim with exec_method: openai_compatible_api.
• Configuration occurs through YAML seat definitions in files like configs/provider-model-slots.nim.example.yaml, specifying the model ID and https://integrate.api.nvidia.com/v1 endpoint.
• Requests route via HTTP with ephemeral Bearer token authentication; keys are never persisted or logged.
• The integration supports model tiering (high/mid) and enables access to open-weight models without local infrastructure.

Frequently Asked Questions

How does CoHI authenticate requests to NVIDIA NIM?

The coordinator reads the API key from the environment variable specified in the seat's api_key_env field (typically NVIDIA_API_KEY) and injects it as an Authorization: Bearer <token> header. As implemented in SKILL.md, this key exists only in memory during the request lifecycle and is excluded from all logging and transcript generation.

Which open-weight models are accessible through NIM integration?

The NVIDIA NIM integration provides access to models including DeepSeek-V4-Pro, Kimi-V1, MiniMax-Text-01, GLM-4, Qwen-2.5, and Nemotron-4. The specific model identifier must match the path format used in NVIDIA's model catalog (e.g., deepseek-ai/deepseek-v4-pro).

How does routing differ between NIM seats and local agents?

NIM seats use exec_method: openai_compatible_api, which triggers HTTP dispatch to the base_url endpoint. Local agents instead execute as sub-processes or local API calls. This distinction is handled automatically by the coordinator based on the provider configuration in SKILL.gemini.md lines 139‑148.

Can I mix NVIDIA NIM seats with other providers in the same council?

Yes. CoHI explicitly supports heterogeneous councils mixing NIM seats with OpenAI, Anthropic, Google, and other providers. This configuration increases response diversity and mitigates shared-training-data bias, as documented in the example configuration comments.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}