# How to Add a New LLM Provider (Ollama, Gemini, or Custom) to the Hiring Agent Pipeline

> Learn to seamlessly integrate new LLM providers like Ollama or Gemini into your Hiring Agent pipeline. Follow our guide to implement the LLMProvider protocol and update prompt and LLM utility files.

- Repository: [HackerRank/hiring-agent](https://github.com/interviewstreet/hiring-agent)
- Tags: how-to-guide
- Published: 2026-07-05

---

**To add a new LLM provider to the hiring-agent pipeline, implement the `LLMProvider` protocol in [`models.py`](https://github.com/interviewstreet/hiring-agent/blob/main/models.py), register the model-to-provider mapping in [`prompt.py`](https://github.com/interviewstreet/hiring-agent/blob/main/prompt.py), and extend the `initialize_llm_provider` function in [`llm_utils.py`](https://github.com/interviewstreet/hiring-agent/blob/main/llm_utils.py) to instantiate your class based on the model name.**

The `interviewstreet/hiring-agent` repository abstracts LLM access behind a provider interface that selects concrete implementations at runtime. This architecture allows you to integrate services like Ollama, Gemini, or any custom API by implementing three specific integration points without modifying the core evaluation logic in [`evaluator.py`](https://github.com/interviewstreet/hiring-agent/blob/main/evaluator.py) or data extraction modules.

## Implement the Provider Class in [`models.py`](https://github.com/interviewstreet/hiring-agent/blob/main/models.py)

Create a provider class that conforms to the `LLMProvider` protocol by implementing the `chat(...)` method. According to the source code, the method must accept `model`, `messages`, `options`, and `**kwargs`, and return a normalized dictionary with the schema `{"message": {"role": "assistant", "content": "..."}}`.

Use the existing implementations as templates:

- **`OllamaProvider`** at lines 71-87 in [`models.py`](https://github.com/interviewstreet/hiring-agent/blob/main/models.py)
- **`GeminiProvider`** at lines 122-152 in [`models.py`](https://github.com/interviewstreet/hiring-agent/blob/main/models.py)

```python

# models.py

class MyNewProvider:
    """Custom LLM provider implementation."""
    
    def __init__(self, api_key: str = ""):
        # Import and configure the SDK for the new service

        import mynewapi as newapi
        self.client = newapi.Client(api_key=api_key)

    def chat(
        self,
        model: str,
        messages: List[Dict[str, str]],
        options: Dict[str, Any] = None,
        **kwargs,
    ) -> Dict[str, Any]:
        """Send a chat request and return an Ollama‑compatible dict."""
        # Convert the generic `messages` list to the provider’s request format

        request = self._to_provider_format(messages, options)
        response = self.client.chat(model=model, **request)

        # Normalise to {"message": {"role": "assistant", "content": "..."}}

        return {"message": {"role": "assistant", "content": response.text}}

```

## Extend the `ModelProvider` Enum (Optional)

If your new service requires a distinct enum value, add it to the `ModelProvider` enum defined at lines 6-11 in [`models.py`](https://github.com/interviewstreet/hiring-agent/blob/main/models.py):

```python

# models.py

class ModelProvider(Enum):
    OLLAMA = "ollama"
    GEMINI = "gemini"
    MYNEW = "mynew"          # ← new entry

```

## Register the Model-to-Provider Mapping in [`prompt.py`](https://github.com/interviewstreet/hiring-agent/blob/main/prompt.py)

Map the model names that should use your new provider in the `MODEL_PROVIDER_MAPPING` dictionary located at lines 46-64 in [`prompt.py`](https://github.com/interviewstreet/hiring-agent/blob/main/prompt.py):

```python

# prompt.py

MODEL_PROVIDER_MAPPING = {
    # … existing entries …

    "mynew‑large": ModelProvider.MYNEW,   # ← new model → new provider

}

```

If you are adding support for a new model that uses an existing provider (for example, a new Ollama model), map it to `ModelProvider.OLLAMA` instead of creating a new enum value.

## Update the Provider Initializer in [`llm_utils.py`](https://github.com/interviewstreet/hiring-agent/blob/main/llm_utils.py)

Extend the `initialize_llm_provider` function at lines 40-63 in [`llm_utils.py`](https://github.com/interviewstreet/hiring-agent/blob/main/llm_utils.py) to instantiate your class when the mapped model is requested. The function reads `MODEL_PROVIDER_MAPPING` and falls back to `OllamaProvider` by default.

```python

# llm_utils.py

def initialize_llm_provider(model_name: str) -> Any:
    provider = OllamaProvider()                     # default

    model_provider = MODEL_PROVIDER_MAPPING.get(
        model_name, ModelProvider.OLLAMA
    )
    if model_provider == ModelProvider.GEMINI:
        # existing Gemini handling …

    elif model_provider == ModelProvider.MYNEW:    # ← new branch

        provider = MyNewProvider(api_key=os.getenv("MYNEW_API_KEY", ""))
    # else keep default Ollama

    return provider

```

## Configure Environment Variables

If the new SDK requires an API key, expose it via `.env.example` and read it in the provider’s `__init__` method. Follow the existing pattern shown in [`prompt.py`](https://github.com/interviewstreet/hiring-agent/blob/main/prompt.py) at lines 66-68 and in `GeminiProvider` at lines 16-20 in [`models.py`](https://github.com/interviewstreet/hiring-agent/blob/main/models.py).

```bash

# .env.example

MYNEW_API_KEY=your_api_key_here

```

## Verify Integration Across the Pipeline

The provider is consumed automatically wherever the pipeline calls `initialize_llm_provider`. You do not need to modify these call sites:

- **Evaluation engine**: [`evaluator.py`](https://github.com/interviewstreet/hiring-agent/blob/main/evaluator.py) at lines 34-38
- **PDF generation**: [`pdf.py`](https://github.com/interviewstreet/hiring-agent/blob/main/pdf.py) at lines 23 and 41-45
- **GitHub data extraction**: [`github.py`](https://github.com/interviewstreet/hiring-agent/blob/main/github.py) at lines 370-372

Run the pipeline with your new model name to verify:

```bash
export DEFAULT_MODEL="mynew-large"
python -m main.evaluator

```

The console output will indicate which provider is selected via the `logger.info` statements inside `initialize_llm_provider`.

## Summary

- **Implement the protocol**: Create a class in [`models.py`](https://github.com/interviewstreet/hiring-agent/blob/main/models.py) with a `chat()` method that returns `{"message": {"role": "assistant", "content": "..."}}`
- **Map the model**: Add entries to `MODEL_PROVIDER_MAPPING` in [`prompt.py`](https://github.com/interviewstreet/hiring-agent/blob/main/prompt.py) to associate model names with your provider
- **Wire the initializer**: Add an `elif` branch in `initialize_llm_provider` ( [`llm_utils.py`](https://github.com/interviewstreet/hiring-agent/blob/main/llm_utils.py) lines 40-63) to instantiate your class
- **Configure secrets**: Add API keys to `.env.example` and read them via `os.getenv()` in your provider's constructor
- **Test transparently**: The rest of the codebase in [`evaluator.py`](https://github.com/interviewstreet/hiring-agent/blob/main/evaluator.py), [`pdf.py`](https://github.com/interviewstreet/hiring-agent/blob/main/pdf.py), and [`github.py`](https://github.com/interviewstreet/hiring-agent/blob/main/github.py) will use your provider automatically when the mapped model name is requested

## Frequently Asked Questions

### What interface must a new LLM provider implement?

A new provider must implement the `chat(self, model, messages, options=None, **kwargs)` method and return a dictionary formatted as `{"message": {"role": "assistant", "content": "..."}}`. This normalization allows the rest of the hiring-agent pipeline to treat all providers interchangeably, as seen in the `OllamaProvider` and `GeminiProvider` implementations in [`models.py`](https://github.com/interviewstreet/hiring-agent/blob/main/models.py).

### Where is the mapping between model names and providers defined?

The mapping is defined in the `MODEL_PROVIDER_MAPPING` dictionary in [`prompt.py`](https://github.com/interviewstreet/hiring-agent/blob/main/prompt.py) at lines 46-64. This dictionary maps string model names (like `"gemini-pro"` or `"llama2"`) to `ModelProvider` enum values, which `initialize_llm_provider` in [`llm_utils.py`](https://github.com/interviewstreet/hiring-agent/blob/main/llm_utils.py) uses to select the correct implementation.

### Do I need to modify the evaluation logic to use a new provider?

No. The evaluation logic in [`evaluator.py`](https://github.com/interviewstreet/hiring-agent/blob/main/evaluator.py) (lines 34-38), PDF generation in [`pdf.py`](https://github.com/interviewstreet/hiring-agent/blob/main/pdf.py) (lines 41-45), and GitHub extraction in [`github.py`](https://github.com/interviewstreet/hiring-agent/blob/main/github.py) (lines 370-372) all call `initialize_llm_provider(DEFAULT_MODEL)`. As long as you register your provider in the mapping and initializer, these modules will use your new provider transparently without code changes.

### How do I handle API keys for a custom provider?

Store the API key in your `.env` file (add it to `.env.example` for documentation), then read it inside your provider's `__init__` method using `os.getenv()`, following the pattern used by `GeminiProvider` at lines 16-20 in [`models.py`](https://github.com/interviewstreet/hiring-agent/blob/main/models.py) and the environment handling at lines 66-68 in [`prompt.py`](https://github.com/interviewstreet/hiring-agent/blob/main/prompt.py).