# How to Use NVIDIA NIM Hosted Models with CodeWhale: A Complete Configuration Guide

> Configure NVIDIA NIM hosted models with CodeWhale for DeepSeek V4. Run LLMs via OpenAI-compatible chat completions using this comprehensive guide.

- Repository: [Hunter Bown/CodeWhale](https://github.com/Hmbown/CodeWhale)
- Tags: how-to-guide
- Published: 2026-06-02

---

**CodeWhale treats NVIDIA NIM as a first-class provider, allowing you to run DeepSeek V4 models through NVIDIA's hosted inference endpoints using standard OpenAI-compatible chat completions.**

CodeWhale is a Rust-based AI coding assistant that integrates multiple LLM providers through a unified abstraction layer. When you configure NVIDIA NIM hosted models with CodeWhale, the system automatically handles authentication, base URL resolution, and model selection while exposing DeepSeek V4 capabilities through both CLI and TUI interfaces.

## Configuring NVIDIA NIM Authentication

CodeWhale resolves API credentials for NVIDIA NIM using a hierarchical lookup strategy defined in [`crates/tui/src/config.rs`](https://github.com/Hmbown/CodeWhale/blob/main/crates/tui/src/config.rs) and documented in [`docs/PROVIDERS.md`](https://github.com/Hmbown/CodeWhale/blob/main/docs/PROVIDERS.md).

**Primary authentication methods:**

1. **Stored credentials** (recommended) – Run `codewhale auth set --provider nvidia-nim` to securely store your key in the system keyring.
2. **Environment variables** – Set `NVIDIA_NIM_API_KEY` or `NVIDIA_API_KEY` (the latter for compatibility with existing NVIDIA tooling).

The authentication logic checks for stored keys first, then falls back to environment variables. This precedence ensures that explicit user configuration overrides temporary environment settings.

```bash

# Store credentials permanently

codewhale auth set --provider nvidia-nim

# Enter your API key when prompted

# Or use environment variables for session-based access

export NVIDIA_NIM_API_KEY=sk-xxxxxxxxxxxxxxxxxxxx

```

## Setting the Base URL and Model

In [`crates/tui/src/config.rs`](https://github.com/Hmbown/CodeWhale/blob/main/crates/tui/src/config.rs), CodeWhale defines default constants for NVIDIA NIM connectivity that you can override through environment variables or configuration files.

**Default configuration values:**
- **Base URL**: `https://integrate.api.nvidia.com/v1` (defined in `DEFAULT_NVIDIA_NIM_BASE_URL`)
- **Default model**: `deepseek-ai/deepseek-v4-pro` (defined in `DEFAULT_NVIDIA_NIM_MODEL`)
- **Flash variant**: `deepseek-ai/deepseek-v4-flash` (defined in `DEFAULT_NVIDIA_NIM_FLASH_MODEL`)

**Environment variable overrides:**
- `NVIDIA_NIM_BASE_URL`, `NIM_BASE_URL`, or `NVIDIA_BASE_URL` for custom endpoints
- `NVIDIA_NIM_MODEL` for selecting alternative models

The `provider_capability` function in [`crates/tui/src/config.rs`](https://github.com/Hmbown/CodeWhale/blob/main/crates/tui/src/config.rs) marks NVIDIA NIM as supporting **cache telemetry** and **reasoning** capabilities, enabling advanced monitoring features when these models are active.

## Using NVIDIA NIM in Practice

### Command-Line Interface

Execute one-off prompts using the `--provider` flag:

```bash
codewhale --provider nvidia-nim "Refactor this Rust function to use iterators instead of loops"

```

### Persistent Configuration

Create or edit `~/.codewhale/config.toml` to set NVIDIA NIM as your default provider:

```toml
provider = "nvidia-nim"

[providers.nvidia_nim]
api_key = "sk-xxxxxxxxxxxxxxxxxxxx"
base_url = "https://integrate.api.nvidia.com/v1"
model = "deepseek-ai/deepseek-v4-pro"

```

After saving, all `codewhale` invocations automatically route to NVIDIA NIM without requiring CLI flags.

### Interactive TUI Selection

Inside CodeWhale's terminal UI:
1. Press `Ctrl-P` to open the provider picker
2. Select **"NVIDIA NIM"** from the list
3. Confirm the **NIM** chip appears in the status line

Alternatively, type `/provider nvidia-nim` in the TUI command palette.

### Environment-Based Configuration

For CI/CD pipelines or temporary testing:

```bash
export NVIDIA_NIM_API_KEY=sk-xxxxxxxxxxxxxxxxxxxx
export NVIDIA_NIM_MODEL=deepseek-ai/deepseek-v4-flash

codewhale "Explain the algorithmic complexity of merge sort"

```

## Technical Architecture

The NVIDIA NIM integration relies on three architectural layers implemented in [`crates/tui/src/config.rs`](https://github.com/Hmbown/CodeWhale/blob/main/crates/tui/src/config.rs):

**Provider Enumeration**
The `ApiProvider::NvidiaNim` enum variant registers the provider ID `nvidia-nim` that the CLI, TUI, and configuration parser recognize as a valid backend option.

**Model Registry**
[`crates/agent/src/lib.rs`](https://github.com/Hmbown/CodeWhale/blob/main/crates/agent/src/lib.rs) maintains a static `ModelRegistry` listing the available NVIDIA NIM models (`deepseek-ai/deepseek-v4-pro` and `deepseek-ai/deepseek-v4-flash`). This registry powers the `codewhale model list` and `codewhale model resolve` commands.

**Request Flow**
[`crates/tui/src/client.rs`](https://github.com/Hmbown/CodeWhale/blob/main/crates/tui/src/client.rs) constructs OpenAI-compatible `/v1/chat/completions` requests using the resolved base URL and authentication headers. Because NVIDIA NIM implements the standard OpenAI API format, no special payload transformation is required beyond the provider-specific configuration.

## Summary

- **CodeWhale** implements NVIDIA NIM support through the `ApiProvider::NvidiaNim` enum in [`crates/tui/src/config.rs`](https://github.com/Hmbown/CodeWhale/blob/main/crates/tui/src/config.rs), treating it as a native inference backend.
- **Authentication** accepts `NVIDIA_NIM_API_KEY` or `NVIDIA_API_KEY` environment variables, or credentials stored via `codewhale auth set`.
- **Default endpoint** points to `https://integrate.api.nvidia.com/v1` with `deepseek-ai/deepseek-v4-pro` as the standard model, though both URL and model ID are configurable.
- **Capabilities** include cache telemetry and reasoning modes, as indicated by the `provider_capability` function metadata.
- **Configuration methods** include CLI flags, environment variables, TUI selection, and persistent `~/.codewhale/config.toml` settings.

## Frequently Asked Questions

### What API key format does CodeWhale expect for NVIDIA NIM?

CodeWhale accepts standard NVIDIA NIM API keys starting with the prefix provided by the NVIDIA Developer Portal. You can set these using `codewhale auth set --provider nvidia-nim` for secure storage, or via the `NVIDIA_NIM_API_KEY` environment variable for temporary access.

### Can I use a custom NVIDIA NIM endpoint instead of the default?

Yes. While CodeWhale defaults to `https://integrate.api.nvidia.com/v1`, you can override this by setting the `NVIDIA_NIM_BASE_URL`, `NIM_BASE_URL`, or `NVIDIA_BASE_URL` environment variable, or by specifying `base_url` in the `[providers.nvidia_nim]` section of your [`config.toml`](https://github.com/Hmbown/CodeWhale/blob/main/config.toml) file.

### Which models are available when using NVIDIA NIM with CodeWhale?

According to the static model registry in [`crates/agent/src/lib.rs`](https://github.com/Hmbown/CodeWhale/blob/main/crates/agent/src/lib.rs), CodeWhale supports `deepseek-ai/deepseek-v4-pro` (the default) and `deepseek-ai/deepseek-v4-flash` (a faster variant). You can select between these using the `NVIDIA_NIM_MODEL` environment variable or the `model` configuration key.

### Does CodeWhale support streaming responses from NVIDIA NIM?

Yes. Because [`crates/tui/src/client.rs`](https://github.com/Hmbown/CodeWhale/blob/main/crates/tui/src/client.rs) uses the standard OpenAI-compatible chat completions protocol to communicate with NVIDIA NIM endpoints, streaming responses work identically to other providers. The client handles Server-Sent Events (SSE) parsing transparently, displaying tokens as they arrive in both CLI and TUI modes.