How to Use NVIDIA NIM Hosted Models with CodeWhale: A Complete Configuration Guide

CodeWhale treats NVIDIA NIM as a first-class provider, allowing you to run DeepSeek V4 models through NVIDIA's hosted inference endpoints using standard OpenAI-compatible chat completions.

CodeWhale is a Rust-based AI coding assistant that integrates multiple LLM providers through a unified abstraction layer. When you configure NVIDIA NIM hosted models with CodeWhale, the system automatically handles authentication, base URL resolution, and model selection while exposing DeepSeek V4 capabilities through both CLI and TUI interfaces.

Configuring NVIDIA NIM Authentication

CodeWhale resolves API credentials for NVIDIA NIM using a hierarchical lookup strategy defined in crates/tui/src/config.rs and documented in docs/PROVIDERS.md.

Primary authentication methods:

1. Stored credentials (recommended) – Run codewhale auth set --provider nvidia-nim to securely store your key in the system keyring.
2. Environment variables – Set NVIDIA_NIM_API_KEY or NVIDIA_API_KEY (the latter for compatibility with existing NVIDIA tooling).

The authentication logic checks for stored keys first, then falls back to environment variables. This precedence ensures that explicit user configuration overrides temporary environment settings.

# Store credentials permanently

codewhale auth set --provider nvidia-nim

# Enter your API key when prompted

# Or use environment variables for session-based access

export NVIDIA_NIM_API_KEY=sk-xxxxxxxxxxxxxxxxxxxx

Setting the Base URL and Model

In crates/tui/src/config.rs, CodeWhale defines default constants for NVIDIA NIM connectivity that you can override through environment variables or configuration files.

Default configuration values:

• Base URL: https://integrate.api.nvidia.com/v1 (defined in DEFAULT_NVIDIA_NIM_BASE_URL)
• Default model: deepseek-ai/deepseek-v4-pro (defined in DEFAULT_NVIDIA_NIM_MODEL)
• Flash variant: deepseek-ai/deepseek-v4-flash (defined in DEFAULT_NVIDIA_NIM_FLASH_MODEL)

Environment variable overrides:

• NVIDIA_NIM_BASE_URL, NIM_BASE_URL, or NVIDIA_BASE_URL for custom endpoints
• NVIDIA_NIM_MODEL for selecting alternative models

The provider_capability function in crates/tui/src/config.rs marks NVIDIA NIM as supporting cache telemetry and reasoning capabilities, enabling advanced monitoring features when these models are active.

Using NVIDIA NIM in Practice

Command-Line Interface

Execute one-off prompts using the --provider flag:

codewhale --provider nvidia-nim "Refactor this Rust function to use iterators instead of loops"

Persistent Configuration

Create or edit ~/.codewhale/config.toml to set NVIDIA NIM as your default provider:

provider = "nvidia-nim"

[providers.nvidia_nim]
api_key = "sk-xxxxxxxxxxxxxxxxxxxx"
base_url = "https://integrate.api.nvidia.com/v1"
model = "deepseek-ai/deepseek-v4-pro"

After saving, all codewhale invocations automatically route to NVIDIA NIM without requiring CLI flags.

Interactive TUI Selection

Inside CodeWhale's terminal UI:

1. Press Ctrl-P to open the provider picker
2. Select "NVIDIA NIM" from the list
3. Confirm the NIM chip appears in the status line

Alternatively, type /provider nvidia-nim in the TUI command palette.

Environment-Based Configuration

For CI/CD pipelines or temporary testing:

export NVIDIA_NIM_API_KEY=sk-xxxxxxxxxxxxxxxxxxxx
export NVIDIA_NIM_MODEL=deepseek-ai/deepseek-v4-flash

codewhale "Explain the algorithmic complexity of merge sort"

Technical Architecture

The NVIDIA NIM integration relies on three architectural layers implemented in crates/tui/src/config.rs:

Provider Enumeration The ApiProvider::NvidiaNim enum variant registers the provider ID nvidia-nim that the CLI, TUI, and configuration parser recognize as a valid backend option.

Model Registry crates/agent/src/lib.rs maintains a static ModelRegistry listing the available NVIDIA NIM models (deepseek-ai/deepseek-v4-pro and deepseek-ai/deepseek-v4-flash). This registry powers the codewhale model list and codewhale model resolve commands.

Request Flow crates/tui/src/client.rs constructs OpenAI-compatible /v1/chat/completions requests using the resolved base URL and authentication headers. Because NVIDIA NIM implements the standard OpenAI API format, no special payload transformation is required beyond the provider-specific configuration.

Summary

• CodeWhale implements NVIDIA NIM support through the ApiProvider::NvidiaNim enum in crates/tui/src/config.rs, treating it as a native inference backend.
• Authentication accepts NVIDIA_NIM_API_KEY or NVIDIA_API_KEY environment variables, or credentials stored via codewhale auth set.
• Default endpoint points to https://integrate.api.nvidia.com/v1 with deepseek-ai/deepseek-v4-pro as the standard model, though both URL and model ID are configurable.
• Capabilities include cache telemetry and reasoning modes, as indicated by the provider_capability function metadata.
• Configuration methods include CLI flags, environment variables, TUI selection, and persistent ~/.codewhale/config.toml settings.

Frequently Asked Questions

What API key format does CodeWhale expect for NVIDIA NIM?

CodeWhale accepts standard NVIDIA NIM API keys starting with the prefix provided by the NVIDIA Developer Portal. You can set these using codewhale auth set --provider nvidia-nim for secure storage, or via the NVIDIA_NIM_API_KEY environment variable for temporary access.

Can I use a custom NVIDIA NIM endpoint instead of the default?

Yes. While CodeWhale defaults to https://integrate.api.nvidia.com/v1, you can override this by setting the NVIDIA_NIM_BASE_URL, NIM_BASE_URL, or NVIDIA_BASE_URL environment variable, or by specifying base_url in the [providers.nvidia_nim] section of your config.toml file.

Which models are available when using NVIDIA NIM with CodeWhale?

According to the static model registry in crates/agent/src/lib.rs, CodeWhale supports deepseek-ai/deepseek-v4-pro (the default) and deepseek-ai/deepseek-v4-flash (a faster variant). You can select between these using the NVIDIA_NIM_MODEL environment variable or the model configuration key.

Does CodeWhale support streaming responses from NVIDIA NIM?

Yes. Because crates/tui/src/client.rs uses the standard OpenAI-compatible chat completions protocol to communicate with NVIDIA NIM endpoints, streaming responses work identically to other providers. The client handles Server-Sent Events (SSE) parsing transparently, displaying tokens as they arrive in both CLI and TUI modes.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}