# How free-claude-code Routes Requests to NVIDIA NIM, OpenRouter, DeepSeek, and LM Studio

> Learn how free-claude-code routes requests to NVIDIA NIM, OpenRouter, DeepSeek, and LM Studio using a dynamic FastAPI system. Converts Claude messages to OpenAI streaming calls.

- Repository: [Ali Khokhar/free-claude-code](https://github.com/Alishahryar1/free-claude-code)
- Tags: internals
- Published: 2026-04-24

---

**The free-claude-code proxy uses a FastAPI-based three-layer routing system that parses model prefixes to dynamically instantiate provider-specific clients, converting Claude-compatible `/v1/messages` requests into OpenAI-compatible streaming calls for each backend.**

The free-claude-code repository implements a unified proxy layer that exposes a Claude-compatible API surface while dynamically routing traffic to diverse LLM backends. This architecture allows developers to use a single client interface while leveraging specialized providers like NVIDIA NIM, OpenRouter, DeepSeek, and LM Studio under the hood. The routing mechanism relies on model string parsing, lazy provider instantiation via a factory pattern, and a shared OpenAI-compatible streaming interface.

## Three-Layer Routing Architecture

The proxy implements routing through three distinct layers, each handling a specific aspect of request translation and forwarding.

### Layer 1: Model String Parsing

Request routing begins in [`config/settings.py`](https://github.com/Alishahryar1/free-claude-code/blob/main/config/settings.py) where the `Settings.parse_provider_type` method extracts the provider prefix from the model identifier. The function splits the incoming model string (or `resolved_provider_model`) on the "/" character and returns the first segment as the provider type.

```python

# From config/settings.py

provider_type = Settings.parse_provider_type(
    request_data.resolved_provider_model or settings.model
)

```

Valid prefixes include `nvidia_nim`, `open_router`, `deepseek`, and `lmstudio`. This prefix determines which concrete provider class handles the request.

### Layer 2: Provider Factory

The `get_provider_for_type` function in [`api/dependencies.py`](https://github.com/Alishahryar1/free-claude-code/blob/main/api/dependencies.py) acts as a factory that lazily creates and caches provider instances. Based on the parsed prefix, it instantiates one of four concrete providers:

- **NvidiaNimProvider**: Configured with NIM API keys and optional proxy settings
- **OpenRouterProvider**: Configured with OpenRouter API keys and reasoning hooks
- **DeepSeekProvider**: Thin wrapper forwarding requests unchanged
- **LMStudioProvider**: Configured to point at local LM Studio endpoints

Each provider receives a `ProviderConfig` object containing base URLs, API keys, and provider-specific knobs injected from environment settings.

### Layer 3: Unified Streaming Interface

All providers inherit from `OpenAICompatibleProvider` defined in [`providers/openai_compat.py`](https://github.com/Alishahryar1/free-claude-code/blob/main/providers/openai_compat.py). This base class wraps the official `AsyncOpenAI` client and implements the Claude-compatible streaming protocol using Server-Sent Events (SSE).

The base class provides a hook method `_build_request_body` that each provider overrides to customize the JSON payload sent to the upstream service. After building the request, the base class calls `self._client.chat.completions.create(..., stream=True)` and translates OpenAI-style chunks into Claude-compatible events using the `SSEBuilder` utility.

## Provider-Specific Routing Implementations

Each backend requires specific request handling and error management.

### NVIDIA NIM Routing

The `NvidiaNimProvider` class in [`providers/nvidia_nim/client.py`](https://github.com/Alishahryar1/free-claude-code/blob/main/providers/nvidia_nim/client.py) extends the base OpenAI-compatible client with NIM-specific request building logic located in [`providers/nvidia_nim/request.py`](https://github.com/Alishahryar1/free-claude-code/blob/main/providers/nvidia_nim/request.py). 

**Key routing characteristics:**
- Automatically strips `reasoning_budget` and `chat_template` parameters from retries when the upstream returns HTTP 400 errors
- Injects NVIDIA-specific metadata into request headers
- Handles model paths formatted as `nvidia_nim/meta/llama-3.1-70b-instruct`

### OpenRouter Routing

The `OpenRouterProvider` in [`providers/open_router/client.py`](https://github.com/Alishahryar1/free-claude-code/blob/main/providers/open_router/client.py) adds specialized handling for OpenRouter's extended reasoning capabilities.

**Key routing characteristics:**
- Renders OpenRouter's `reasoning_details` field as Claude "thinking" events in the SSE stream
- Maps model identifiers from the `open_router/` prefix to OpenRouter's internal model strings
- Builds request bodies via [`providers/open_router/request.py`](https://github.com/Alishahryar1/free-claude-code/blob/main/providers/open_router/request.py) to handle OpenRouter-specific parameters

### DeepSeek Routing

The `DeepSeekProvider` in [`providers/deepseek/client.py`](https://github.com/Alishahryar1/free-claude-code/blob/main/providers/deepseek/client.py) functions as a minimal pass-through wrapper around the OpenAI-compatible client.

**Key routing characteristics:**
- Forwards requests without transformation using the standard OpenAI client configuration
- Expects model identifiers prefixed with `deepseek/`
- Relies on the base class for all streaming and error handling

### LM Studio Routing

The `LMStudioProvider` routes requests to locally running LM Studio instances rather than cloud APIs.

**Key routing characteristics:**
- Configures `ProviderConfig` to use `settings.lm_studio_base_url` (typically `http://localhost:1234`)
- Uses a dummy API key value of `"lm-studio"` for local authentication
- Expects model identifiers prefixed with `lmstudio/`

## Complete Request Flow

When a client sends a POST request to `/v1/messages`, the following routing sequence executes:

1. **FastAPI Entry Point**: The `create_message` function in [`api/routes.py`](https://github.com/Alishahryar1/free-claude-code/blob/main/api/routes.py) receives the request and extracts the provider type using `Settings.parse_provider_type`.

2. **Provider Instantiation**: The factory function `get_provider_for_type` in [`api/dependencies.py`](https://github.com/Alishahryar1/free-claude-code/blob/main/api/dependencies.py) creates the appropriate provider instance based on the parsed prefix.

3. **Request Building**: The provider's `_build_request_body` method constructs the JSON payload specific to the target service (NIM fields for NVIDIA, reasoning hooks for OpenRouter, etc.).

4. **Upstream Streaming**: The base class calls the OpenAI-compatible endpoint with `stream=True` and processes chunks through [`providers/common/SSEBuilder.py`](https://github.com/Alishahryar1/free-claude-code/blob/main/providers/common/SSEBuilder.py).

5. **Client Response**: `create_message` wraps the async generator in a FastAPI `StreamingResponse` with `text/event-stream` MIME type, delivering Claude-compatible events.

## Practical Routing Examples

Route requests to specific providers using the model prefix notation:

### Routing to NVIDIA NIM

```bash
curl -X POST https://my-proxy.example.com/v1/messages \
  -H "Content-Type: application/json" \
  -H "x-api-key: $PROXY_AUTH_TOKEN" \
  -d '{
        "model": "nvidia_nim/meta/llama-3.1-70b-instruct",
        "messages": [{"role":"user","content":"Write a Python hello world"}]
      }' \
  --no-buffer

```

The `nvidia_nim/` prefix triggers `parse_provider_type` to select the NVIDIA NIM provider, which handles model-specific parameters and retry logic for NIM endpoints.

### Routing to OpenRouter

```bash
curl -X POST https://my-proxy.example.com/v1/messages \
  -H "Content-Type: application/json" \
  -H "x-api-key: $PROXY_AUTH_TOKEN" \
  -d '{
        "model": "open_router/deepseek/deepseek-chat",
        "messages": [{"role":"user","content":"Explain recursion"}]
      }' \
  --no-buffer

```

The `open_router/` prefix instantiates `OpenRouterProvider`, which maps the request to OpenRouter's model catalog and handles reasoning detail extraction.

### Routing to DeepSeek

```bash
curl -X POST https://my-proxy.example.com/v1/messages \
  -H "Content-Type: application/json" \
  -H "x-api-key: $PROXY_AUTH_TOKEN" \
  -d '{
        "model": "deepseek/deepseek-chat",
        "messages": [{"role":"user","content":"Analyze this code"}]
      }' \
  --no-buffer

```

The `deepseek/` prefix routes through `DeepSeekProvider` with no request transformation, using standard OpenAI client configuration.

### Routing to LM Studio

```bash
curl -X POST https://my-proxy.example.com/v1/messages \
  -H "Content-Type: application/json" \
  -H "x-api-key: $PROXY_AUTH_TOKEN" \
  -d '{
        "model": "lmstudio/qwen2.5-7b",
        "messages": [{"role":"user","content":"Summarize the last paragraph"}]
      }' \
  --no-buffer

```

The `lmstudio/` prefix selects `LMStudioProvider`, which forwards the request to the local LM Studio server configured in `settings.lm_studio_base_url`.

## Summary

- **Prefix-based routing**: The `parse_provider_type` function in [`config/settings.py`](https://github.com/Alishahryar1/free-claude-code/blob/main/config/settings.py) extracts provider identifiers from model strings using "/" as a delimiter.
- **Factory pattern**: `get_provider_for_type` in [`api/dependencies.py`](https://github.com/Alishahryar1/free-claude-code/blob/main/api/dependencies.py) instantiates specialized providers (`NvidiaNimProvider`, `OpenRouterProvider`, `DeepSeekProvider`, `LMStudioProvider`) with appropriate configurations.
- **Unified interface**: All providers inherit from `OpenAICompatibleProvider` in [`providers/openai_compat.py`](https://github.com/Alishahryar1/free-claude-code/blob/main/providers/openai_compat.py), ensuring consistent Claude-compatible streaming output regardless of the upstream backend.
- **Provider customization**: Each provider implements specific request building and error handling (NIM strips invalid parameters on retry, OpenRouter handles reasoning details, LM Studio uses local endpoints).

## Frequently Asked Questions

### How does the proxy determine which provider to use for a request?

The proxy extracts the provider prefix from the model string by splitting on the "/" character. For example, a model name of `nvidia_nim/meta/llama-3.1-70b-instruct` returns `nvidia_nim` as the provider type, which the factory uses to instantiate the corresponding client class.

### What happens if a provider-specific parameter causes an error?

The `NvidiaNimProvider` implements specific retry logic that catches HTTP 400 responses and automatically strips problematic parameters like `reasoning_budget` or `chat_template` before retrying the request. This handling is implemented in [`providers/nvidia_nim/client.py`](https://github.com/Alishahryar1/free-claude-code/blob/main/providers/nvidia_nim/client.py).

### Can I run the proxy against a local LM Studio instance?

Yes. Set the `lm_studio_base_url` setting to your local LM Studio endpoint (typically `http://localhost:1234`), then prefix your model names with `lmstudio/`. The `LMStudioProvider` automatically uses a dummy API key of `"lm-studio"` and routes requests to your local server instead of cloud APIs.

### How does the proxy handle streaming responses differently across providers?

All providers use the shared `OpenAICompatibleProvider` base class, which standardizes streaming through [`providers/common/SSEBuilder.py`](https://github.com/Alishahryar1/free-claude-code/blob/main/providers/common/SSEBuilder.py). Provider-specific implementations only customize the request payload construction via `_build_request_body`, while the base class handles the SSE formatting and Claude-compatible event generation consistently across all backends.