# How to Integrate Additional LLM Providers Beyond Ollama and Gemini in Hiring Agent

> Learn to integrate additional LLM providers into the hiring agent by extending the ModelProvider enum implementing a provider class and updating prompt.py.

- Repository: [HackerRank/hiring-agent](https://github.com/interviewstreet/hiring-agent)
- Tags: how-to-guide
- Published: 2026-07-10

---

**The hiring-agent repository abstracts LLMs behind a unified `LLMProvider` protocol, allowing you to add new providers by extending the `ModelProvider` enum in [`models.py`](https://github.com/interviewstreet/hiring-agent/blob/main/models.py), implementing a provider class with a `chat()` method, updating the `MODEL_PROVIDER_MAPPING` dictionary in [`prompt.py`](https://github.com/interviewstreet/hiring-agent/blob/main/prompt.py), and wiring the factory in `initialize_llm_provider`.**

The interviewstreet/hiring-agent project uses a provider-based architecture to support multiple Large Language Model backends. While it ships with built-in support for Ollama and Google Gemini, the modular design makes it straightforward to integrate additional LLM providers beyond Ollama and Gemini without modifying the core evaluation logic.

## Architecture Overview

The system relies on four key components that work together to route requests to the appropriate backend:

- **`ModelProvider` enum** – Defined in [`models.py`](https://github.com/interviewstreet/hiring-agent/blob/main/models.py) at line 6, this enum enumerates supported backends. Currently includes `OLLAMA` and `GEMINI` values that you can extend with new entries.

- **Provider classes** – Concrete implementations like `OllamaProvider` (line 71) and `GeminiProvider` (line 13) in [`models.py`](https://github.com/interviewstreet/hiring-agent/blob/main/models.py) that wrap vendor SDKs and normalize responses to a common format.

- **`MODEL_PROVIDER_MAPPING`** – Located in [`prompt.py`](https://github.com/interviewstreet/hiring-agent/blob/main/prompt.py) at line 46, this dictionary maps specific model names (e.g., `gemini-2.5-pro`) to their corresponding `ModelProvider` enum values.

- **`initialize_llm_provider`** – Factory function in [`llm_utils.py`](https://github.com/interviewstreet/hiring-agent/blob/main/llm_utils.py) at line 40 that instantiates the correct provider class based on the model name, falling back to Ollama when necessary.

When you integrate additional LLM providers beyond Ollama and Gemini, these four touchpoints ensure the rest of the codebase—including score calculation and PDF handling—continues to function without modification.

## Step-by-Step Implementation Guide

Follow these steps to add a new LLM backend to the system.

### Step 1: Extend the ModelProvider Enum

Add a new enum value to the `ModelProvider` class in [`main/models.py`](https://github.com/interviewstreet/hiring-agent/blob/main/main/models.py) to represent your provider:

```python
from enum import Enum

class ModelProvider(Enum):
    OLLAMA = "ollama"
    GEMINI = "gemini"
    OPENAI = "openai"  # New entry for your provider

```

### Step 2: Implement the Provider Class

Create a provider class in [`main/models.py`](https://github.com/interviewstreet/hiring-agent/blob/main/main/models.py) that implements the `LLMProvider` protocol. The class must expose a `chat(self, model, messages, options=None, **kwargs)` method that returns a dictionary with a normalized response structure:

```python
from typing import List, Dict, Any

class OpenAIProvider:
    """OpenAI API provider implementation."""
    
    def __init__(self, api_key: str):
        import openai
        openai.api_key = api_key
        self.client = openai

    def chat(
        self,
        model: str,
        messages: List[Dict[str, str]],
        options: Dict[str, Any] = None,
        **kwargs,
    ) -> Dict[str, Any]:
        # Convert generic message format to provider-specific format

        payload = {"model": model, "messages": messages}
        if options:
            payload.update(options)  # e.g., temperature, top_p

        
        response = self.client.ChatCompletion.create(**payload)
        
        # Normalize to Ollama-like dict expected by downstream code

        return {
            "message": {
                "role": "assistant",
                "content": response.choices[0].message.content
            }
        }

```

### Step 3: Update the Model-to-Provider Mapping

Register your model names in [`main/prompt.py`](https://github.com/interviewstreet/hiring-agent/blob/main/main/prompt.py) by adding entries to `MODEL_PROVIDER_MAPPING`:

```python
from main.models import ModelProvider

MODEL_PROVIDER_MAPPING = {
    # Existing entries...

    "gpt-4o-mini": ModelProvider.OPENAI,
    "gpt-4o": ModelProvider.OPENAI,
}

```

### Step 4: Configure Environment Variables

Add any required API keys or configuration variables to [`main/prompt.py`](https://github.com/interviewstreet/hiring-agent/blob/main/main/prompt.py) alongside existing environment variable definitions:

```python
import os

OPENAI_API_KEY = os.getenv("OPENAI_API_KEY", "")

```

### Step 5: Wire the Factory Function

Update the `initialize_llm_provider` function in [`main/llm_utils.py`](https://github.com/interviewstreet/hiring-agent/blob/main/main/llm_utils.py) to instantiate your new provider when the corresponding enum value is detected:

```python
from main.prompt import MODEL_PROVIDER_MAPPING, OPENAI_API_KEY
from main.models import ModelProvider, OpenAIProvider
import logging

logger = logging.getLogger(__name__)

def initialize_llm_provider(model_name: str) -> Any:
    model_provider = MODEL_PROVIDER_MAPPING.get(model_name)
    
    if model_provider == ModelProvider.GEMINI:
        # Existing Gemini initialization...

        from main.models import GeminiProvider
        return GeminiProvider()
    
    elif model_provider == ModelProvider.OPENAI:
        if not OPENAI_API_KEY:
            logger.warning("OpenAI API key not found. Falling back to Ollama.")
            # Fall through to default Ollama behavior

        else:
            logger.info(f"Using OpenAI provider with model {model_name}")
            return OpenAIProvider(api_key=OPENAI_API_KEY)
    
    # Default Ollama branch

    from main.models import OllamaProvider
    return OllamaProvider()

```

## Key Files for LLM Integration

When you integrate additional LLM providers beyond Ollama and Gemini, these three files constitute the complete integration surface:

- **[`main/models.py`](https://github.com/interviewstreet/hiring-agent/blob/main/main/models.py)** – Contains the `ModelProvider` enum and provider class definitions (`OllamaProvider`, `GeminiProvider`, and your new implementations).

- **[`main/prompt.py`](https://github.com/interviewstreet/hiring-agent/blob/main/main/prompt.py)** – Houses the `MODEL_PROVIDER_MAPPING` dictionary, default model configuration, and environment variable loading for API keys.

- **[`main/llm_utils.py`](https://github.com/interviewstreet/hiring-agent/blob/main/main/llm_utils.py)** – Implements the `initialize_llm_provider` factory function that routes model names to the appropriate provider instances.

## Summary

- **Provider Protocol**: The system expects a `chat()` method returning a normalized dictionary with a `message` key containing `role` and `content`.

- **Four Touchpoints**: Enum extension, class implementation, mapping registration, and factory wiring are the only changes required to integrate additional LLM providers beyond Ollama and Gemini.

- **Zero Downstream Changes**: After implementation, existing evaluation pipelines automatically route requests through your new provider when configured via `DEFAULT_MODEL` or specific model name parameters.

- **Environment Configuration**: Store sensitive credentials in environment variables loaded through [`prompt.py`](https://github.com/interviewstreet/hiring-agent/blob/main/prompt.py) to maintain security consistency with existing providers.

## Frequently Asked Questions

### How does the system handle provider fallback if my new LLM is unavailable?

The `initialize_llm_provider` function in [`main/llm_utils.py`](https://github.com/interviewstreet/hiring-agent/blob/main/main/llm_utils.py) typically defaults to Ollama when a provider initialization fails or when API keys are missing. You can implement similar fallback logic by checking for missing credentials and logging warnings before falling through to the default Ollama branch.

### Can I add cloud providers like Azure OpenAI or Anthropic Claude using the same pattern?

Yes. The architecture supports any LLM service that offers a Python SDK. Implement the `chat()` method to translate the generic message format into provider-specific API calls, then normalize the response to match the `{"message": {"role": "assistant", "content": "..."}}` structure expected by the evaluation pipeline.

### Where should I store API keys for new providers?

Store API keys as environment variables in [`main/prompt.py`](https://github.com/interviewstreet/hiring-agent/blob/main/main/prompt.py) following the pattern used for existing providers. Import these variables into [`main/llm_utils.py`](https://github.com/interviewstreet/hiring-agent/blob/main/main/llm_utils.py) within the `initialize_llm_provider` function to pass them to your provider class constructor.

### Do I need to modify the resume evaluation logic when adding a new provider?

No. The evaluation logic in interviewstreet/hiring-agent calls `initialize_llm_provider` and interacts with the returned object only through the `LLMProvider` protocol. As long as your provider implements the `chat()` method with the correct signature and return format, the scoring and PDF processing modules require no changes.