# How the Hiring Agent Supports Switching Between Different LLM Providers

> Learn how the Hiring Agent effortlessly switches LLM providers. Discover its clever factory pattern, defaulting to Ollama and conditionally routing to Google Gemini.

- Repository: [HackerRank/hiring-agent](https://github.com/interviewstreet/hiring-agent)
- Tags: how-to-guide
- Published: 2026-07-09

---

**The Hiring Agent supports switching between different LLM providers through a configuration-driven factory pattern that maps model names to specific provider implementations, defaulting to local Ollama inference while conditionally routing to Google Gemini when API credentials are present.**

The `interviewstreet/hiring-agent` repository abstracts the underlying large-language-model service behind a uniform interface. This architecture allows core business logic—such as resume parsing and candidate scoring—to remain unchanged while the actual model provider is swapped at runtime based on configuration.

## Configuration-Driven Provider Mapping

The switching mechanism begins with a static mapping defined in **[`prompt.py`](https://github.com/interviewstreet/hiring-agent/blob/main/prompt.py)**. The `MODEL_PROVIDER_MAPPING` dictionary associates user-requested model identifiers with specific `ModelProvider` enum values.

```python

# Defined in prompt.py

MODEL_PROVIDER_MAPPING = {
    "gemini-1.5-pro": ModelProvider.GEMINI,
    "gemini-1.5-flash": ModelProvider.GEMINI,
    # Local Ollama models typically omit explicit mapping

}

class ModelProvider(Enum):
    OLLAMA = "ollama"
    GEMINI = "gemini"

```

When a model name is not present in this mapping, the system defaults to **`ModelProvider.OLLAMA`**, ensuring local inference works out-of-the-box without explicit configuration.

## The Provider Factory Implementation

The **`initialize_llm_provider`** function in **[`llm_utils.py`](https://github.com/interviewstreet/hiring-agent/blob/main/llm_utils.py)** serves as the central factory that instantiates the correct provider class at runtime.

```python
def initialize_llm_provider(model_name: str) -> Any:
    """
    Choose the proper LLM provider (Ollama or Gemini) based on the
    supplied model name.
    """
    provider = OllamaProvider()
    model_provider = MODEL_PROVIDER_MAPPING.get(model_name, ModelProvider.OLLAMA)

    if model_provider == ModelProvider.GEMINI:
        if not GEMINI_API_KEY:
            logger.warning("⚠️ Gemini API key not found. Falling back to Ollama.")
        else:
            logger.info(f"🔄 Using Google Gemini API provider with model {model_name}")
            provider = GeminiProvider(api_key=GEMINI_API_KEY)
    else:
        logger.info(f"🔄 Using Ollama provider with model {model_name}")

    return provider

```

**Key aspects of this implementation:**

- **Fail-Safe Defaulting:** The function initializes with `OllamaProvider()` as the default before checking configuration, ensuring the application remains functional even if Gemini credentials are missing.
- **Environment Validation:** When the mapping resolves to `ModelProvider.GEMINI`, the function validates the presence of the **`GEMINI_API_KEY`** environment variable before instantiation.
- **Graceful Degradation:** If the Gemini API key is absent, the function logs a warning and returns the Ollama provider, preventing runtime crashes while alerting operators to the configuration issue.

## Unified Interface Architecture

Both providers implement the **`LLMProvider`** protocol defined in **[`models.py`](https://github.com/interviewstreet/hiring-agent/blob/main/models.py)**, exposing a consistent `chat` method signature that accepts `model`, `messages`, and optional `options` parameters.

```python

# models.py

class LLMProvider(Protocol):
    def chat(self, model: str, messages: List[Dict], options: Optional[Dict] = None) -> Dict:
        ...

```

**Concrete implementations:**

- **`OllamaProvider`:** Communicates with a local Ollama daemon via HTTP, ideal for air-gapped environments or cost-sensitive deployments.
- **`GeminiProvider`:** Wraps the Google Gemini REST API, handling authentication headers and request formatting transparently.

This protocol-based design means consuming code calls `provider.chat()` without import-time dependencies on specific vendor SDKs, achieving true provider-agnostic operation.

## Runtime Provider Selection Example

The following pattern demonstrates how the same codebase switches between local and cloud inference without modification to business logic:

```python
from llm_utils import initialize_llm_provider
from prompt import DEFAULT_MODEL  # e.g., "gemini-1.5-pro" or "llama3.1-8b"

# 1. Factory selects provider based on model name

provider = initialize_llm_provider(DEFAULT_MODEL)

# 2. Prepare standardized message payload

messages = [
    {"role": "system", "content": "You are a helpful hiring assistant."},
    {"role": "user", "content": "Summarize this resume."}
]

# 3. Unified interface executes regardless of underlying provider

response = provider.chat(
    model=DEFAULT_MODEL,
    messages=messages,
    options={"temperature": 0.7}
)

content = response["message"]["content"]

```

If `DEFAULT_MODEL` maps to a Gemini identifier and `GEMINI_API_KEY` is set, the request routes to Google's API; otherwise, it automatically falls back to the local Ollama instance.

## Summary

- **Configuration Mapping:** The `MODEL_PROVIDER_MAPPING` dictionary in [`prompt.py`](https://github.com/interviewstreet/hiring-agent/blob/main/prompt.py) binds model names to `ModelProvider` enum values, driving the selection logic.
- **Factory Pattern:** `initialize_llm_provider()` in [`llm_utils.py`](https://github.com/interviewstreet/hiring-agent/blob/main/llm_utils.py) encapsulates instantiation logic, defaulting to `OllamaProvider` while conditionally creating `GeminiProvider` when credentials exist.
- **Protocol Abstraction:** Both providers implement the `LLMProvider` protocol from [`models.py`](https://github.com/interviewstreet/hiring-agent/blob/main/models.py), exposing identical `chat()` signatures that abstract vendor-specific implementations.
- **Environment-Based Switching:** The system reads `GEMINI_API_KEY` at runtime to determine cloud eligibility, requiring zero code changes to switch between local and hosted inference.

## Frequently Asked Questions

### How do I add support for a new LLM provider like OpenAI?

Extend the `ModelProvider` enum in [`models.py`](https://github.com/interviewstreet/hiring-agent/blob/main/models.py) with a new member (e.g., `OPENAI = "openai"`), create a concrete class implementing the `LLMProvider` protocol with a `chat()` method, and update `MODEL_PROVIDER_MAPPING` in [`prompt.py`](https://github.com/interviewstreet/hiring-agent/blob/main/prompt.py) to map specific model names to your new enum value. Finally, modify `initialize_llm_provider` in [`llm_utils.py`](https://github.com/interviewstreet/hiring-agent/blob/main/llm_utils.py) to instantiate your new class when that enum is detected.

### Why does the system default to Ollama instead of failing when Gemini is unavailable?

The architecture prioritizes **fail-safe operation** for hiring workflows that may run in secure environments without internet access. By defaulting to Ollama, the system ensures the application remains functional for resume processing and candidate evaluation even when external API credentials are not configured, while logging warnings to indicate the fallback state.

### Can I force the system to use a specific provider regardless of the model name?

Yes, you can bypass the mapping logic by instantiating the provider class directly. Instead of calling `initialize_llm_provider()`, import `OllamaProvider` or `GeminiProvider` from [`models.py`](https://github.com/interviewstreet/hiring-agent/blob/main/models.py) and instantiate them directly with the required parameters. However, this sacrifices the runtime flexibility that the factory pattern provides.

### Where should I store the Gemini API key for the Hiring Agent to detect it?

The system expects the Gemini API key in the **`GEMINI_API_KEY`** environment variable, which is imported in [`prompt.py`](https://github.com/interviewstreet/hiring-agent/blob/main/prompt.py) and checked within `initialize_llm_provider()`. Set this variable in your shell environment or `.env` file before starting the application; if the variable is undefined or empty, the system automatically falls back to Ollama.