# How the Provider Factory in free-claude-code Creates and Manages Provider Instances

> Discover how the free-claude-code provider factory uses singleton caching to efficiently create and manage AI provider instances, reusing HTTP clients and rate limiters for optimal performance.

- Repository: [Ali Khokhar/free-claude-code](https://github.com/Alishahryar1/free-claude-code)
- Tags: internals
- Published: 2026-04-24

---

**The provider factory in free-claude-code implements a singleton caching mechanism in [`api/dependencies.py`](https://github.com/Alishahryar1/free-claude-code/blob/main/api/dependencies.py) that instantiates AI provider objects on demand and reuses them across the application lifecycle to share HTTP clients and rate limiters.**

The `Alishahryar1/free-claude-code` repository provides a unified interface for multiple AI backends, abstracting provider-specific complexity behind a common base class. Its **provider factory** pattern centralizes configuration management, credential validation, and resource pooling, ensuring that each provider type is instantiated exactly once and shared across all API requests.

## Global Cache and Factory Entry Point

At the heart of the factory lies a module-level cache defined in [`api/dependencies.py`](https://github.com/Alishahryar1/free-claude-code/blob/main/api/dependencies.py):

```python
_providers: dict[str, BaseProvider] = {}

```

This dictionary stores singleton instances of each concrete provider, keyed by provider type strings such as `"nvidia_nim"` or `"open_router"`. The public interface `get_provider_for_type()` implements the retrieval logic:

```python
def get_provider_for_type(provider_type: str) -> BaseProvider:
    """Get or create a provider for the given provider type."""
    if provider_type not in _providers:
        _providers[provider_type] = _create_provider_for_type(
            provider_type, get_settings()
        )
    return _providers[provider_type]

```

When the requested type is absent from the cache, the factory delegates to `_create_provider_for_type()` to build and store the instance. Subsequent invocations return the cached object, guaranteeing that expensive resources—such as HTTP connection pools and rate-limiting state—remain shared across concurrent requests.

## Dynamic Provider Construction

The private function `_create_provider_for_type()` handles the dynamic assembly of provider objects. It constructs a **`ProviderConfig`** (defined in [`providers/base.py`](https://github.com/Alishahryar1/free-claude-code/blob/main/providers/base.py)) containing shared settings like API keys, timeouts, and proxy configurations:

```python
def _create_provider_for_type(provider_type: str, settings: Settings) -> BaseProvider:
    # Resolve optional proxy per-type

    _proxy_map = {
        "nvidia_nim": _get_proxy_value(settings, "nvidia_nim_proxy"),
        "open_router": _get_proxy_value(settings, "open_router_proxy"),
        "lmstudio": _get_proxy_value(settings, "lmstudio_proxy"),
        "llamacpp": _get_proxy_value(settings, "llamacpp_proxy"),
    }
    proxy = _proxy_map.get(provider_type, "")

```

### NVIDIA NIM Provider Instantiation

For the `"nvidia_nim"` type, the factory validates the API key before instantiation:

```python
if provider_type == "nvidia_nim":
    if not settings.nvidia_nim_api_key or not settings.nvidia_nim_api_key.strip():
        raise AuthenticationError("NVIDIA_NIM_API_KEY is required")
    config = ProviderConfig(
        api_key=settings.nvidia_nim_api_key,
        base_url=NVIDIA_NIM_BASE_URL,
        rate_limit=settings.provider_rate_limit,
        rate_window=settings.provider_rate_window,
        max_concurrency=settings.provider_max_concurrency,
        http_read_timeout=settings.http_read_timeout,
        http_write_timeout=settings.http_write_timeout,
        http_connect_timeout=settings.http_connect_timeout,
        enable_thinking=settings.enable_thinking,
        proxy=proxy,
    )
    return NvidiaNimProvider(config, nim_settings=settings.nim)

```

The resulting `NvidiaNimProvider` (implemented in [`providers/nvidia_nim.py`](https://github.com/Alishahryar1/free-claude-code/blob/main/providers/nvidia_nim.py)) receives both the shared configuration and provider-specific settings.

### OpenRouter and Other Providers

The factory follows identical patterns for additional backends:

- **OpenRouter**: Validates `OPEN_ROUTER_API_KEY` and instantiates `OpenRouterProvider` from [`providers/open_router/__init__.py`](https://github.com/Alishahryar1/free-claude-code/blob/main/providers/open_router/__init__.py)
- **LMStudio**: Configures `LMStudioProvider` from [`providers/lmstudio.py`](https://github.com/Alishahryar1/free-claude-code/blob/main/providers/lmstudio.py)
- **LlamaCPP**: Builds `LlamaCppProvider` from [`providers/llamacpp/__init__.py`](https://github.com/Alishahryar1/free-claude-code/blob/main/providers/llamacpp/__init__.py)
- **DeepSeek**: Creates `DeepSeekProvider` with appropriate credential checks

If an unrecognized `provider_type` is supplied, the factory raises a `ValueError` after logging the error.

## Default Provider Resolution

For convenience, the `get_provider()` function provides the default provider based on the `MODEL` environment variable:

```python
def get_provider() -> BaseProvider:
    """Get or create the default provider (based on MODEL env var)."""
    return get_provider_for_type(get_settings().provider_type)

```

This allows FastAPI endpoints to depend on a provider without specifying the type explicitly.

## Resource Cleanup and Lifecycle Management

When the application shuts down, `cleanup_provider()` ensures graceful resource release:

```python
async def cleanup_provider():
    global _providers
    for provider in _providers.values():
        await provider.cleanup()
    _providers = {}

```

This async function iterates through all cached instances, invoking their individual `cleanup()` methods to close HTTP clients and release connections, then clears the cache dictionary.

## Practical Usage Examples

### Injecting Providers into FastAPI Routes

Use FastAPI's dependency injection to receive a configured provider instance:

```python
from fastapi import APIRouter, Depends
from api.dependencies import get_provider

router = APIRouter()

@router.post("/v1/chat/completions")
async def chat_completion(request: Any, provider=Depends(get_provider)):
    """
    Receives a cached provider instance (e.g., NvidiaNimProvider)
    based on the MODEL environment variable.
    """
    async for chunk in provider.stream_response(request):
        yield chunk

```

### Manually Retrieving Specific Providers

Bypass the default configuration to obtain a specific provider type:

```python
from api.dependencies import get_provider_for_type

# Obtain LMStudio provider regardless of global MODEL setting

lmstudio = get_provider_for_type("lmstudio")

# Verify singleton behavior: same instance returned

assert lmstudio is get_provider_for_type("lmstudio")

```

### Verifying Caching Behavior

Demonstrate that the factory returns identical instances:

```python
from api.dependencies import _providers, get_provider_for_type

# Cache initially empty

assert "nvidia_nim" not in _providers

# First call creates and caches

provider1 = get_provider_for_type("nvidia_nim")
assert "nvidia_nim" in _providers

# Second call returns cached instance

provider2 = get_provider_for_type("nvidia_nim")
assert provider1 is provider2  # Same object identity

```

## Summary

- The **provider factory** in [`api/dependencies.py`](https://github.com/Alishahryar1/free-claude-code/blob/main/api/dependencies.py) uses a global `_providers` dictionary to implement singleton caching for all AI provider types.
- **`get_provider_for_type()`** serves as the primary entry point, lazily instantiating providers via **`_create_provider_for_type()`** when cache misses occur.
- Each provider receives a **`ProviderConfig`** object constructed from environment settings, including API keys, timeouts, rate limits, and proxy configurations.
- Supported providers include **NVIDIA NIM**, **OpenRouter**, **DeepSeek**, **LMStudio**, and **LlamaCPP**, each implemented in dedicated modules under `providers/`.
- The **`cleanup_provider()`** async function ensures proper resource disposal during application shutdown by invoking `cleanup()` on each cached instance.

## Frequently Asked Questions

### How does the provider factory handle API key validation?

The factory validates provider-specific API keys during instantiation within `_create_provider_for_type()`. For example, when `provider_type` is `"nvidia_nim"`, the code checks `settings.nvidia_nim_api_key`, raising an `AuthenticationError` if the key is missing or empty. Each provider branch implements similar validation logic before constructing the `ProviderConfig`.

### Can I use multiple provider types simultaneously in the same application?

Yes. The `_providers` cache maintains separate instances for each provider type string, allowing you to use `get_provider_for_type("nvidia_nim")` and `get_provider_for_type("open_router")` in the same process. Each type is instantiated once and cached independently, enabling concurrent access to different AI backends while sharing resources within each provider type.

### Where should I configure proxy settings for specific providers?

Proxy configurations are resolved in `_create_provider_for_type()` within [`api/dependencies.py`](https://github.com/Alishahryar1/free-claude-code/blob/main/api/dependencies.py). The function maps provider type strings to specific proxy environment variables (such as `nvidia_nim_proxy` or `open_router_proxy`) using the `_proxy_map` dictionary. These values are then injected into the `ProviderConfig` passed to the provider constructor.

### What happens to provider resources during application shutdown?

The async **`cleanup_provider()`** function iterates through all values in the global `_providers` dictionary and calls `await provider.cleanup()` on each instance. This ensures HTTP clients and other network resources are properly released before the application exits, preventing connection leaks and graceful termination of pending requests.