How the Provider Factory in free-claude-code Creates and Manages Provider Instances

The provider factory in free-claude-code implements a singleton caching mechanism in api/dependencies.py that instantiates AI provider objects on demand and reuses them across the application lifecycle to share HTTP clients and rate limiters.

The Alishahryar1/free-claude-code repository provides a unified interface for multiple AI backends, abstracting provider-specific complexity behind a common base class. Its provider factory pattern centralizes configuration management, credential validation, and resource pooling, ensuring that each provider type is instantiated exactly once and shared across all API requests.

Global Cache and Factory Entry Point

At the heart of the factory lies a module-level cache defined in api/dependencies.py:

_providers: dict[str, BaseProvider] = {}

This dictionary stores singleton instances of each concrete provider, keyed by provider type strings such as "nvidia_nim" or "open_router". The public interface get_provider_for_type() implements the retrieval logic:

def get_provider_for_type(provider_type: str) -> BaseProvider:
    """Get or create a provider for the given provider type."""
    if provider_type not in _providers:
        _providers[provider_type] = _create_provider_for_type(
            provider_type, get_settings()
        )
    return _providers[provider_type]

When the requested type is absent from the cache, the factory delegates to _create_provider_for_type() to build and store the instance. Subsequent invocations return the cached object, guaranteeing that expensive resources—such as HTTP connection pools and rate-limiting state—remain shared across concurrent requests.

Dynamic Provider Construction

The private function _create_provider_for_type() handles the dynamic assembly of provider objects. It constructs a ProviderConfig (defined in providers/base.py) containing shared settings like API keys, timeouts, and proxy configurations:

def _create_provider_for_type(provider_type: str, settings: Settings) -> BaseProvider:
    # Resolve optional proxy per-type

    _proxy_map = {
        "nvidia_nim": _get_proxy_value(settings, "nvidia_nim_proxy"),
        "open_router": _get_proxy_value(settings, "open_router_proxy"),
        "lmstudio": _get_proxy_value(settings, "lmstudio_proxy"),
        "llamacpp": _get_proxy_value(settings, "llamacpp_proxy"),
    }
    proxy = _proxy_map.get(provider_type, "")

NVIDIA NIM Provider Instantiation

For the "nvidia_nim" type, the factory validates the API key before instantiation:

if provider_type == "nvidia_nim":
    if not settings.nvidia_nim_api_key or not settings.nvidia_nim_api_key.strip():
        raise AuthenticationError("NVIDIA_NIM_API_KEY is required")
    config = ProviderConfig(
        api_key=settings.nvidia_nim_api_key,
        base_url=NVIDIA_NIM_BASE_URL,
        rate_limit=settings.provider_rate_limit,
        rate_window=settings.provider_rate_window,
        max_concurrency=settings.provider_max_concurrency,
        http_read_timeout=settings.http_read_timeout,
        http_write_timeout=settings.http_write_timeout,
        http_connect_timeout=settings.http_connect_timeout,
        enable_thinking=settings.enable_thinking,
        proxy=proxy,
    )
    return NvidiaNimProvider(config, nim_settings=settings.nim)

The resulting NvidiaNimProvider (implemented in providers/nvidia_nim.py) receives both the shared configuration and provider-specific settings.

OpenRouter and Other Providers

The factory follows identical patterns for additional backends:

• OpenRouter: Validates OPEN_ROUTER_API_KEY and instantiates OpenRouterProvider from providers/open_router/__init__.py
• LMStudio: Configures LMStudioProvider from providers/lmstudio.py
• LlamaCPP: Builds LlamaCppProvider from providers/llamacpp/__init__.py
• DeepSeek: Creates DeepSeekProvider with appropriate credential checks

If an unrecognized provider_type is supplied, the factory raises a ValueError after logging the error.

Default Provider Resolution

For convenience, the get_provider() function provides the default provider based on the MODEL environment variable:

def get_provider() -> BaseProvider:
    """Get or create the default provider (based on MODEL env var)."""
    return get_provider_for_type(get_settings().provider_type)

This allows FastAPI endpoints to depend on a provider without specifying the type explicitly.

Resource Cleanup and Lifecycle Management

When the application shuts down, cleanup_provider() ensures graceful resource release:

async def cleanup_provider():
    global _providers
    for provider in _providers.values():
        await provider.cleanup()
    _providers = {}

This async function iterates through all cached instances, invoking their individual cleanup() methods to close HTTP clients and release connections, then clears the cache dictionary.

Practical Usage Examples

Injecting Providers into FastAPI Routes

Use FastAPI's dependency injection to receive a configured provider instance:

from fastapi import APIRouter, Depends
from api.dependencies import get_provider

router = APIRouter()

@router.post("/v1/chat/completions")
async def chat_completion(request: Any, provider=Depends(get_provider)):
    """
    Receives a cached provider instance (e.g., NvidiaNimProvider)
    based on the MODEL environment variable.
    """
    async for chunk in provider.stream_response(request):
        yield chunk

Manually Retrieving Specific Providers

Bypass the default configuration to obtain a specific provider type:

from api.dependencies import get_provider_for_type

# Obtain LMStudio provider regardless of global MODEL setting

lmstudio = get_provider_for_type("lmstudio")

# Verify singleton behavior: same instance returned

assert lmstudio is get_provider_for_type("lmstudio")

Verifying Caching Behavior

Demonstrate that the factory returns identical instances:

from api.dependencies import _providers, get_provider_for_type

# Cache initially empty

assert "nvidia_nim" not in _providers

# First call creates and caches

provider1 = get_provider_for_type("nvidia_nim")
assert "nvidia_nim" in _providers

# Second call returns cached instance

provider2 = get_provider_for_type("nvidia_nim")
assert provider1 is provider2  # Same object identity

Summary

• The provider factory in api/dependencies.py uses a global _providers dictionary to implement singleton caching for all AI provider types.
• get_provider_for_type() serves as the primary entry point, lazily instantiating providers via _create_provider_for_type() when cache misses occur.
• Each provider receives a ProviderConfig object constructed from environment settings, including API keys, timeouts, rate limits, and proxy configurations.
• Supported providers include NVIDIA NIM, OpenRouter, DeepSeek, LMStudio, and LlamaCPP, each implemented in dedicated modules under providers/.
• The cleanup_provider() async function ensures proper resource disposal during application shutdown by invoking cleanup() on each cached instance.

Frequently Asked Questions

How does the provider factory handle API key validation?

The factory validates provider-specific API keys during instantiation within _create_provider_for_type(). For example, when provider_type is "nvidia_nim", the code checks settings.nvidia_nim_api_key, raising an AuthenticationError if the key is missing or empty. Each provider branch implements similar validation logic before constructing the ProviderConfig.

Can I use multiple provider types simultaneously in the same application?

Yes. The _providers cache maintains separate instances for each provider type string, allowing you to use get_provider_for_type("nvidia_nim") and get_provider_for_type("open_router") in the same process. Each type is instantiated once and cached independently, enabling concurrent access to different AI backends while sharing resources within each provider type.

Where should I configure proxy settings for specific providers?

Proxy configurations are resolved in _create_provider_for_type() within api/dependencies.py. The function maps provider type strings to specific proxy environment variables (such as nvidia_nim_proxy or open_router_proxy) using the _proxy_map dictionary. These values are then injected into the ProviderConfig passed to the provider constructor.

What happens to provider resources during application shutdown?

The async cleanup_provider() function iterates through all values in the global _providers dictionary and calls await provider.cleanup() on each instance. This ensures HTTP clients and other network resources are properly released before the application exits, preventing connection leaks and graceful termination of pending requests.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}