How to Add a New LLM Provider to the Hiring Agent

To add a new LLM provider to the hiring agent, implement the LLMProvider protocol in a new provider class, register the model-to-provider mapping in prompt.py, and extend the initialize_llm_provider factory in llm_utils.py to instantiate the provider at runtime.

The hiring agent from the interviewstreet/hiring-agent repository supports multiple LLM backends through a protocol-based abstraction layer. By following the established integration pattern found in models.py and llm_utils.py, you can add support for OpenAI, Anthropic, Cohere, or any custom API without modifying the core evaluation logic in pdf.py, evaluator.py, or github.py.

Step 1: Create a Provider Class Following the LLMProvider Protocol

Create a new Python file (e.g., openai_provider.py) containing a class that implements the LLMProvider protocol defined in models.py. Your class must implement a chat method with the following signature:

def chat(self, model, messages, options=None, **kwargs) -> Dict

The method must return a dictionary matching the Ollama-compatible response structure: {"message": {"role": "assistant", "content": "..."}}.

Here is the skeleton for a new OpenAI provider:

from typing import List, Dict, Any
from models import LLMProvider

class OpenAIProvider:
    """OpenAI API implementation of the LLMProvider protocol."""
    
    def __init__(self, api_key: str):
        import openai
        openai.api_key = api_key
        self.client = openai

    def chat(self, model: str, messages: List[Dict[str, str]],
             options: Dict[str, Any] = None, **kwargs) -> Dict[str, Any]:
        """Execute chat completion and return Ollama-compatible response."""
        response = self.client.ChatCompletion.create(
            model=model,
            messages=messages,
            **(options or {})
        )
        return {
            "message": {
                "role": "assistant",
                "content": response["choices"][0]["message"]["content"]
            }
        }

Step 2: Extend the ModelProvider Enum

Add a new enum value to ModelProvider in models.py to represent your backend:

from enum import Enum

class ModelProvider(Enum):
    OLLAMA = "ollama"
    GEMINI = "gemini"
    OPENAI = "openai"   # New entry

This enum acts as the discriminator that initialize_llm_provider uses to select the correct implementation at runtime.

Step 3: Register Model-to-Provider Mappings

In prompt.py, update MODEL_PROVIDER_MAPPING to associate specific model names with your new enum value:

MODEL_PROVIDER_MAPPING = {
    "llama3.1": ModelProvider.OLLAMA,
    "gemini-1.5-flash": ModelProvider.GEMINI,
    "gpt-4o-mini": ModelProvider.OPENAI,   # New entry

}

If your provider requires model-specific parameters (temperature, max tokens, etc.), add entries to the MODEL_PARAMETERS dictionary in the same file.

Step 4: Configure Environment Variables

Expose required credentials via environment variables. Add your variable to .env.example:

OPENAI_API_KEY=your_key_here

Then load and validate this variable in prompt.py:

import os
OPENAI_API_KEY = os.getenv("OPENAI_API_KEY")

Step 5: Extend the initialize_llm_provider Factory

Modify initialize_llm_provider in llm_utils.py to instantiate your class when the mapping resolves to your new enum. Import your new provider class and add the conditional logic:

from openai_provider import OpenAIProvider  # Import from your new module

from prompt import OPENAI_API_KEY, GEMINI_API_KEY

def initialize_llm_provider(model_name: str) -> Any:
    provider = OllamaProvider()  # Default fallback

    model_provider = MODEL_PROVIDER_MAPPING.get(model_name, ModelProvider.OLLAMA)
    
    if model_provider == ModelProvider.GEMINI:
        # Existing Gemini handling...

        pass
    elif model_provider == ModelProvider.OPENAI:
        if not OPENAI_API_KEY:
            logger.warning("⚠️ OpenAI API key missing – falling back to Ollama.")
        else:
            logger.info(f"🔄 Using OpenAI provider with model {model_name}")
            provider = OpenAIProvider(api_key=OPENAI_API_KEY)
    else:
        logger.info(f"🔄 Using Ollama provider with model {model_name}")
    
    return provider

Step 6: Update Imports

Ensure llm_utils.py imports the new provider class from the correct module. If you created openai_provider.py, add:

from openai_provider import OpenAIProvider

If you defined the class directly in models.py, import from there instead.

Step 7: Test the Integration

Run the test suite or manually invoke a workflow that triggers initialize_llm_provider with your registered model name. Verify that:

• The provider initializes correctly when the API key is present
• The system gracefully falls back to OllamaProvider when credentials are missing
• The chat method returns properly formatted responses matching the expected structure

Key Files in the LLM Provider Architecture

Understanding these core files ensures you integrate providers correctly without breaking existing functionality:

• models.py – Defines the LLMProvider protocol, the ModelProvider enum, and built-in provider classes (OllamaProvider, GeminiProvider).
• prompt.py – Contains MODEL_PROVIDER_MAPPING and MODEL_PARAMETERS tables; also loads environment variables for API authentication.
• llm_utils.py – Houses initialize_llm_provider, the central factory function that consumers (pdf.py, evaluator.py, github.py) use to obtain an LLM instance.

Summary

Adding a new LLM provider to the hiring agent follows a strict protocol-based integration pattern:

• Implement the LLMProvider protocol with a chat method returning an Ollama-compatible response dictionary
• Register the provider in the ModelProvider enum and MODEL_PROVIDER_MAPPING in prompt.py
• Configure environment variables for authentication and load them in the configuration module
• Extend initialize_llm_provider in llm_utils.py to handle the new enum case and instantiate your class
• Test thoroughly to ensure fallback logic and response formatting work as expected across all consumers

Frequently Asked Questions

What is the LLMProvider protocol in the hiring agent?

The LLMProvider protocol is an interface defined in models.py that requires implementing classes to provide a chat(self, model, messages, options=None, **kwargs) -> Dict method. This standardization allows the hiring agent to switch between Ollama, Gemini, or custom providers without changing calling code in evaluator.py or github.py, as all providers return a uniform response structure.

How does the hiring agent select which LLM provider to use?

The hiring agent selects providers at runtime via the initialize_llm_provider function in llm_utils.py. This factory function looks up the requested model name in MODEL_PROVIDER_MAPPING (defined in prompt.py), instantiates the corresponding provider class (e.g., OpenAIProvider), and returns it to the caller. If credentials are missing or the model is unrecognized, it automatically falls back to OllamaProvider.

Can I integrate Anthropic or Cohere using this same process?

Yes. Any LLM service that offers a Python SDK or HTTP API can be integrated by following the same seven steps: create a provider class implementing the chat method, add an enum value to ModelProvider, map your model names in MODEL_PROVIDER_MAPPING, expose API keys via environment variables, and extend initialize_llm_provider to handle the new enum case with appropriate authentication checks.

What happens if the API key for my new provider is missing?

The initialize_llm_provider function includes defensive logic that checks for the presence of required environment variables (e.g., OPENAI_API_KEY). If the variable is missing or empty, the function logs a warning message ("⚠️ OpenAI API key missing – falling back to Ollama") and returns an OllamaProvider instance instead, ensuring the hiring agent remains functional using local models without crashing.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}