How free-claude-code Routes Requests to NVIDIA NIM, OpenRouter, DeepSeek, and LM Studio

The free-claude-code proxy uses a FastAPI-based three-layer routing system that parses model prefixes to dynamically instantiate provider-specific clients, converting Claude-compatible /v1/messages requests into OpenAI-compatible streaming calls for each backend.

The free-claude-code repository implements a unified proxy layer that exposes a Claude-compatible API surface while dynamically routing traffic to diverse LLM backends. This architecture allows developers to use a single client interface while leveraging specialized providers like NVIDIA NIM, OpenRouter, DeepSeek, and LM Studio under the hood. The routing mechanism relies on model string parsing, lazy provider instantiation via a factory pattern, and a shared OpenAI-compatible streaming interface.

Three-Layer Routing Architecture

The proxy implements routing through three distinct layers, each handling a specific aspect of request translation and forwarding.

Layer 1: Model String Parsing

Request routing begins in config/settings.py where the Settings.parse_provider_type method extracts the provider prefix from the model identifier. The function splits the incoming model string (or resolved_provider_model) on the "/" character and returns the first segment as the provider type.

# From config/settings.py

provider_type = Settings.parse_provider_type(
    request_data.resolved_provider_model or settings.model
)

Valid prefixes include nvidia_nim, open_router, deepseek, and lmstudio. This prefix determines which concrete provider class handles the request.

Layer 2: Provider Factory

The get_provider_for_type function in api/dependencies.py acts as a factory that lazily creates and caches provider instances. Based on the parsed prefix, it instantiates one of four concrete providers:

• NvidiaNimProvider: Configured with NIM API keys and optional proxy settings
• OpenRouterProvider: Configured with OpenRouter API keys and reasoning hooks
• DeepSeekProvider: Thin wrapper forwarding requests unchanged
• LMStudioProvider: Configured to point at local LM Studio endpoints

Each provider receives a ProviderConfig object containing base URLs, API keys, and provider-specific knobs injected from environment settings.

Layer 3: Unified Streaming Interface

All providers inherit from OpenAICompatibleProvider defined in providers/openai_compat.py. This base class wraps the official AsyncOpenAI client and implements the Claude-compatible streaming protocol using Server-Sent Events (SSE).

The base class provides a hook method _build_request_body that each provider overrides to customize the JSON payload sent to the upstream service. After building the request, the base class calls self._client.chat.completions.create(..., stream=True) and translates OpenAI-style chunks into Claude-compatible events using the SSEBuilder utility.

Provider-Specific Routing Implementations

Each backend requires specific request handling and error management.

NVIDIA NIM Routing

The NvidiaNimProvider class in providers/nvidia_nim/client.py extends the base OpenAI-compatible client with NIM-specific request building logic located in providers/nvidia_nim/request.py.

Key routing characteristics:

• Automatically strips reasoning_budget and chat_template parameters from retries when the upstream returns HTTP 400 errors
• Injects NVIDIA-specific metadata into request headers
• Handles model paths formatted as nvidia_nim/meta/llama-3.1-70b-instruct

OpenRouter Routing

The OpenRouterProvider in providers/open_router/client.py adds specialized handling for OpenRouter's extended reasoning capabilities.

Key routing characteristics:

• Renders OpenRouter's reasoning_details field as Claude "thinking" events in the SSE stream
• Maps model identifiers from the open_router/ prefix to OpenRouter's internal model strings
• Builds request bodies via providers/open_router/request.py to handle OpenRouter-specific parameters

DeepSeek Routing

The DeepSeekProvider in providers/deepseek/client.py functions as a minimal pass-through wrapper around the OpenAI-compatible client.

Key routing characteristics:

• Forwards requests without transformation using the standard OpenAI client configuration
• Expects model identifiers prefixed with deepseek/
• Relies on the base class for all streaming and error handling

LM Studio Routing

The LMStudioProvider routes requests to locally running LM Studio instances rather than cloud APIs.

Key routing characteristics:

• Configures ProviderConfig to use settings.lm_studio_base_url (typically http://localhost:1234)
• Uses a dummy API key value of "lm-studio" for local authentication
• Expects model identifiers prefixed with lmstudio/

Complete Request Flow

When a client sends a POST request to /v1/messages, the following routing sequence executes:

1. FastAPI Entry Point: The create_message function in api/routes.py receives the request and extracts the provider type using Settings.parse_provider_type.

2. Provider Instantiation: The factory function get_provider_for_type in api/dependencies.py creates the appropriate provider instance based on the parsed prefix.

3. Request Building: The provider's _build_request_body method constructs the JSON payload specific to the target service (NIM fields for NVIDIA, reasoning hooks for OpenRouter, etc.).

4. Upstream Streaming: The base class calls the OpenAI-compatible endpoint with stream=True and processes chunks through providers/common/SSEBuilder.py.

5. Client Response: create_message wraps the async generator in a FastAPI StreamingResponse with text/event-stream MIME type, delivering Claude-compatible events.

Practical Routing Examples

Route requests to specific providers using the model prefix notation:

Routing to NVIDIA NIM

curl -X POST https://my-proxy.example.com/v1/messages \
  -H "Content-Type: application/json" \
  -H "x-api-key: $PROXY_AUTH_TOKEN" \
  -d '{
        "model": "nvidia_nim/meta/llama-3.1-70b-instruct",
        "messages": [{"role":"user","content":"Write a Python hello world"}]
      }' \
  --no-buffer

The nvidia_nim/ prefix triggers parse_provider_type to select the NVIDIA NIM provider, which handles model-specific parameters and retry logic for NIM endpoints.

Routing to OpenRouter

curl -X POST https://my-proxy.example.com/v1/messages \
  -H "Content-Type: application/json" \
  -H "x-api-key: $PROXY_AUTH_TOKEN" \
  -d '{
        "model": "open_router/deepseek/deepseek-chat",
        "messages": [{"role":"user","content":"Explain recursion"}]
      }' \
  --no-buffer

The open_router/ prefix instantiates OpenRouterProvider, which maps the request to OpenRouter's model catalog and handles reasoning detail extraction.

Routing to DeepSeek

curl -X POST https://my-proxy.example.com/v1/messages \
  -H "Content-Type: application/json" \
  -H "x-api-key: $PROXY_AUTH_TOKEN" \
  -d '{
        "model": "deepseek/deepseek-chat",
        "messages": [{"role":"user","content":"Analyze this code"}]
      }' \
  --no-buffer

The deepseek/ prefix routes through DeepSeekProvider with no request transformation, using standard OpenAI client configuration.

Routing to LM Studio

curl -X POST https://my-proxy.example.com/v1/messages \
  -H "Content-Type: application/json" \
  -H "x-api-key: $PROXY_AUTH_TOKEN" \
  -d '{
        "model": "lmstudio/qwen2.5-7b",
        "messages": [{"role":"user","content":"Summarize the last paragraph"}]
      }' \
  --no-buffer

The lmstudio/ prefix selects LMStudioProvider, which forwards the request to the local LM Studio server configured in settings.lm_studio_base_url.

Summary

• Prefix-based routing: The parse_provider_type function in config/settings.py extracts provider identifiers from model strings using "/" as a delimiter.
• Factory pattern: get_provider_for_type in api/dependencies.py instantiates specialized providers (NvidiaNimProvider, OpenRouterProvider, DeepSeekProvider, LMStudioProvider) with appropriate configurations.
• Unified interface: All providers inherit from OpenAICompatibleProvider in providers/openai_compat.py, ensuring consistent Claude-compatible streaming output regardless of the upstream backend.
• Provider customization: Each provider implements specific request building and error handling (NIM strips invalid parameters on retry, OpenRouter handles reasoning details, LM Studio uses local endpoints).

Frequently Asked Questions

How does the proxy determine which provider to use for a request?

The proxy extracts the provider prefix from the model string by splitting on the "/" character. For example, a model name of nvidia_nim/meta/llama-3.1-70b-instruct returns nvidia_nim as the provider type, which the factory uses to instantiate the corresponding client class.

What happens if a provider-specific parameter causes an error?

The NvidiaNimProvider implements specific retry logic that catches HTTP 400 responses and automatically strips problematic parameters like reasoning_budget or chat_template before retrying the request. This handling is implemented in providers/nvidia_nim/client.py.

Can I run the proxy against a local LM Studio instance?

Yes. Set the lm_studio_base_url setting to your local LM Studio endpoint (typically http://localhost:1234), then prefix your model names with lmstudio/. The LMStudioProvider automatically uses a dummy API key of "lm-studio" and routes requests to your local server instead of cloud APIs.

How does the proxy handle streaming responses differently across providers?

All providers use the shared OpenAICompatibleProvider base class, which standardizes streaming through providers/common/SSEBuilder.py. Provider-specific implementations only customize the request payload construction via _build_request_body, while the base class handles the SSE formatting and Claude-compatible event generation consistently across all backends.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}