How to Integrate Additional LLM Providers Beyond Ollama and Gemini in Hiring Agent

The hiring-agent repository abstracts LLMs behind a unified LLMProvider protocol, allowing you to add new providers by extending the ModelProvider enum in models.py, implementing a provider class with a chat() method, updating the MODEL_PROVIDER_MAPPING dictionary in prompt.py, and wiring the factory in initialize_llm_provider.

The interviewstreet/hiring-agent project uses a provider-based architecture to support multiple Large Language Model backends. While it ships with built-in support for Ollama and Google Gemini, the modular design makes it straightforward to integrate additional LLM providers beyond Ollama and Gemini without modifying the core evaluation logic.

Architecture Overview

The system relies on four key components that work together to route requests to the appropriate backend:

• ModelProvider enum – Defined in models.py at line 6, this enum enumerates supported backends. Currently includes OLLAMA and GEMINI values that you can extend with new entries.

• Provider classes – Concrete implementations like OllamaProvider (line 71) and GeminiProvider (line 13) in models.py that wrap vendor SDKs and normalize responses to a common format.

• MODEL_PROVIDER_MAPPING – Located in prompt.py at line 46, this dictionary maps specific model names (e.g., gemini-2.5-pro) to their corresponding ModelProvider enum values.

• initialize_llm_provider – Factory function in llm_utils.py at line 40 that instantiates the correct provider class based on the model name, falling back to Ollama when necessary.

When you integrate additional LLM providers beyond Ollama and Gemini, these four touchpoints ensure the rest of the codebase—including score calculation and PDF handling—continues to function without modification.

Step-by-Step Implementation Guide

Follow these steps to add a new LLM backend to the system.

Step 1: Extend the ModelProvider Enum

Add a new enum value to the ModelProvider class in main/models.py to represent your provider:

from enum import Enum

class ModelProvider(Enum):
    OLLAMA = "ollama"
    GEMINI = "gemini"
    OPENAI = "openai"  # New entry for your provider

Step 2: Implement the Provider Class

Create a provider class in main/models.py that implements the LLMProvider protocol. The class must expose a chat(self, model, messages, options=None, **kwargs) method that returns a dictionary with a normalized response structure:

from typing import List, Dict, Any

class OpenAIProvider:
    """OpenAI API provider implementation."""
    
    def __init__(self, api_key: str):
        import openai
        openai.api_key = api_key
        self.client = openai

    def chat(
        self,
        model: str,
        messages: List[Dict[str, str]],
        options: Dict[str, Any] = None,
        **kwargs,
    ) -> Dict[str, Any]:
        # Convert generic message format to provider-specific format

        payload = {"model": model, "messages": messages}
        if options:
            payload.update(options)  # e.g., temperature, top_p

        
        response = self.client.ChatCompletion.create(**payload)
        
        # Normalize to Ollama-like dict expected by downstream code

        return {
            "message": {
                "role": "assistant",
                "content": response.choices[0].message.content
            }
        }

Step 3: Update the Model-to-Provider Mapping

Register your model names in main/prompt.py by adding entries to MODEL_PROVIDER_MAPPING:

from main.models import ModelProvider

MODEL_PROVIDER_MAPPING = {
    # Existing entries...

    "gpt-4o-mini": ModelProvider.OPENAI,
    "gpt-4o": ModelProvider.OPENAI,
}

Step 4: Configure Environment Variables

Add any required API keys or configuration variables to main/prompt.py alongside existing environment variable definitions:

import os

OPENAI_API_KEY = os.getenv("OPENAI_API_KEY", "")

Step 5: Wire the Factory Function

Update the initialize_llm_provider function in main/llm_utils.py to instantiate your new provider when the corresponding enum value is detected:

from main.prompt import MODEL_PROVIDER_MAPPING, OPENAI_API_KEY
from main.models import ModelProvider, OpenAIProvider
import logging

logger = logging.getLogger(__name__)

def initialize_llm_provider(model_name: str) -> Any:
    model_provider = MODEL_PROVIDER_MAPPING.get(model_name)
    
    if model_provider == ModelProvider.GEMINI:
        # Existing Gemini initialization...

        from main.models import GeminiProvider
        return GeminiProvider()
    
    elif model_provider == ModelProvider.OPENAI:
        if not OPENAI_API_KEY:
            logger.warning("OpenAI API key not found. Falling back to Ollama.")
            # Fall through to default Ollama behavior

        else:
            logger.info(f"Using OpenAI provider with model {model_name}")
            return OpenAIProvider(api_key=OPENAI_API_KEY)
    
    # Default Ollama branch

    from main.models import OllamaProvider
    return OllamaProvider()

Key Files for LLM Integration

When you integrate additional LLM providers beyond Ollama and Gemini, these three files constitute the complete integration surface:

• main/models.py – Contains the ModelProvider enum and provider class definitions (OllamaProvider, GeminiProvider, and your new implementations).

• main/prompt.py – Houses the MODEL_PROVIDER_MAPPING dictionary, default model configuration, and environment variable loading for API keys.

• main/llm_utils.py – Implements the initialize_llm_provider factory function that routes model names to the appropriate provider instances.

Summary

• Provider Protocol: The system expects a chat() method returning a normalized dictionary with a message key containing role and content.

• Four Touchpoints: Enum extension, class implementation, mapping registration, and factory wiring are the only changes required to integrate additional LLM providers beyond Ollama and Gemini.

• Zero Downstream Changes: After implementation, existing evaluation pipelines automatically route requests through your new provider when configured via DEFAULT_MODEL or specific model name parameters.

• Environment Configuration: Store sensitive credentials in environment variables loaded through prompt.py to maintain security consistency with existing providers.

Frequently Asked Questions

How does the system handle provider fallback if my new LLM is unavailable?

The initialize_llm_provider function in main/llm_utils.py typically defaults to Ollama when a provider initialization fails or when API keys are missing. You can implement similar fallback logic by checking for missing credentials and logging warnings before falling through to the default Ollama branch.

Can I add cloud providers like Azure OpenAI or Anthropic Claude using the same pattern?

Yes. The architecture supports any LLM service that offers a Python SDK. Implement the chat() method to translate the generic message format into provider-specific API calls, then normalize the response to match the {"message": {"role": "assistant", "content": "..."}} structure expected by the evaluation pipeline.

Where should I store API keys for new providers?

Store API keys as environment variables in main/prompt.py following the pattern used for existing providers. Import these variables into main/llm_utils.py within the initialize_llm_provider function to pass them to your provider class constructor.

Do I need to modify the resume evaluation logic when adding a new provider?

No. The evaluation logic in interviewstreet/hiring-agent calls initialize_llm_provider and interacts with the returned object only through the LLMProvider protocol. As long as your provider implements the chat() method with the correct signature and return format, the scoring and PDF processing modules require no changes.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}