# How to Add a New LLM Provider to the Hiring Agent > Learn how to add a new LLM provider to the hiring agent by implementing the LLMProvider protocol, registering mappings, and extending the initialize_llm_provider factory. Integrate new LLM capabilities seamlessly. - Repository: [HackerRank/hiring-agent](https://github.com/interviewstreet/hiring-agent) - Tags: how-to-guide - Published: 2026-07-19 --- **To add a new LLM provider to the hiring agent, implement the `LLMProvider` protocol in a new provider class, register the model-to-provider mapping in [`prompt.py`](https://github.com/interviewstreet/hiring-agent/blob/main/prompt.py), and extend the `initialize_llm_provider` factory in [`llm_utils.py`](https://github.com/interviewstreet/hiring-agent/blob/main/llm_utils.py) to instantiate the provider at runtime.** The hiring agent from the interviewstreet/hiring-agent repository supports multiple LLM backends through a protocol-based abstraction layer. By following the established integration pattern found in [`models.py`](https://github.com/interviewstreet/hiring-agent/blob/main/models.py) and [`llm_utils.py`](https://github.com/interviewstreet/hiring-agent/blob/main/llm_utils.py), you can add support for OpenAI, Anthropic, Cohere, or any custom API without modifying the core evaluation logic in [`pdf.py`](https://github.com/interviewstreet/hiring-agent/blob/main/pdf.py), [`evaluator.py`](https://github.com/interviewstreet/hiring-agent/blob/main/evaluator.py), or [`github.py`](https://github.com/interviewstreet/hiring-agent/blob/main/github.py). ## Step 1: Create a Provider Class Following the LLMProvider Protocol Create a new Python file (e.g., [`openai_provider.py`](https://github.com/interviewstreet/hiring-agent/blob/main/openai_provider.py)) containing a class that implements the `LLMProvider` protocol defined in [`models.py`](https://github.com/interviewstreet/hiring-agent/blob/main/models.py). Your class must implement a `chat` method with the following signature: ```python def chat(self, model, messages, options=None, **kwargs) -> Dict ``` The method must return a dictionary matching the Ollama-compatible response structure: `{"message": {"role": "assistant", "content": "..."}}`. Here is the skeleton for a new OpenAI provider: ```python from typing import List, Dict, Any from models import LLMProvider class OpenAIProvider: """OpenAI API implementation of the LLMProvider protocol.""" def __init__(self, api_key: str): import openai openai.api_key = api_key self.client = openai def chat(self, model: str, messages: List[Dict[str, str]], options: Dict[str, Any] = None, **kwargs) -> Dict[str, Any]: """Execute chat completion and return Ollama-compatible response.""" response = self.client.ChatCompletion.create( model=model, messages=messages, **(options or {}) ) return { "message": { "role": "assistant", "content": response["choices"][0]["message"]["content"] } } ``` ## Step 2: Extend the ModelProvider Enum Add a new enum value to `ModelProvider` in [`models.py`](https://github.com/interviewstreet/hiring-agent/blob/main/models.py) to represent your backend: ```python from enum import Enum class ModelProvider(Enum): OLLAMA = "ollama" GEMINI = "gemini" OPENAI = "openai" # New entry ``` This enum acts as the discriminator that `initialize_llm_provider` uses to select the correct implementation at runtime. ## Step 3: Register Model-to-Provider Mappings In [`prompt.py`](https://github.com/interviewstreet/hiring-agent/blob/main/prompt.py), update `MODEL_PROVIDER_MAPPING` to associate specific model names with your new enum value: ```python MODEL_PROVIDER_MAPPING = { "llama3.1": ModelProvider.OLLAMA, "gemini-1.5-flash": ModelProvider.GEMINI, "gpt-4o-mini": ModelProvider.OPENAI, # New entry } ``` If your provider requires model-specific parameters (temperature, max tokens, etc.), add entries to the `MODEL_PARAMETERS` dictionary in the same file. ## Step 4: Configure Environment Variables Expose required credentials via environment variables. Add your variable to `.env.example`: ```bash OPENAI_API_KEY=your_key_here ``` Then load and validate this variable in [`prompt.py`](https://github.com/interviewstreet/hiring-agent/blob/main/prompt.py): ```python import os OPENAI_API_KEY = os.getenv("OPENAI_API_KEY") ``` ## Step 5: Extend the initialize_llm_provider Factory Modify `initialize_llm_provider` in [`llm_utils.py`](https://github.com/interviewstreet/hiring-agent/blob/main/llm_utils.py) to instantiate your class when the mapping resolves to your new enum. Import your new provider class and add the conditional logic: ```python from openai_provider import OpenAIProvider # Import from your new module from prompt import OPENAI_API_KEY, GEMINI_API_KEY def initialize_llm_provider(model_name: str) -> Any: provider = OllamaProvider() # Default fallback model_provider = MODEL_PROVIDER_MAPPING.get(model_name, ModelProvider.OLLAMA) if model_provider == ModelProvider.GEMINI: # Existing Gemini handling... pass elif model_provider == ModelProvider.OPENAI: if not OPENAI_API_KEY: logger.warning("⚠️ OpenAI API key missing – falling back to Ollama.") else: logger.info(f"🔄 Using OpenAI provider with model {model_name}") provider = OpenAIProvider(api_key=OPENAI_API_KEY) else: logger.info(f"🔄 Using Ollama provider with model {model_name}") return provider ``` ## Step 6: Update Imports Ensure [`llm_utils.py`](https://github.com/interviewstreet/hiring-agent/blob/main/llm_utils.py) imports the new provider class from the correct module. If you created [`openai_provider.py`](https://github.com/interviewstreet/hiring-agent/blob/main/openai_provider.py), add: ```python from openai_provider import OpenAIProvider ``` If you defined the class directly in [`models.py`](https://github.com/interviewstreet/hiring-agent/blob/main/models.py), import from there instead. ## Step 7: Test the Integration Run the test suite or manually invoke a workflow that triggers `initialize_llm_provider` with your registered model name. Verify that: - The provider initializes correctly when the API key is present - The system gracefully falls back to `OllamaProvider` when credentials are missing - The `chat` method returns properly formatted responses matching the expected structure ## Key Files in the LLM Provider Architecture Understanding these core files ensures you integrate providers correctly without breaking existing functionality: - **[`models.py`](https://github.com/interviewstreet/hiring-agent/blob/main/models.py)** – Defines the `LLMProvider` protocol, the `ModelProvider` enum, and built-in provider classes (`OllamaProvider`, `GeminiProvider`). - **[`prompt.py`](https://github.com/interviewstreet/hiring-agent/blob/main/prompt.py)** – Contains `MODEL_PROVIDER_MAPPING` and `MODEL_PARAMETERS` tables; also loads environment variables for API authentication. - **[`llm_utils.py`](https://github.com/interviewstreet/hiring-agent/blob/main/llm_utils.py)** – Houses `initialize_llm_provider`, the central factory function that consumers ([`pdf.py`](https://github.com/interviewstreet/hiring-agent/blob/main/pdf.py), [`evaluator.py`](https://github.com/interviewstreet/hiring-agent/blob/main/evaluator.py), [`github.py`](https://github.com/interviewstreet/hiring-agent/blob/main/github.py)) use to obtain an LLM instance. ## Summary Adding a new LLM provider to the hiring agent follows a strict protocol-based integration pattern: - **Implement** the `LLMProvider` protocol with a `chat` method returning an Ollama-compatible response dictionary - **Register** the provider in the `ModelProvider` enum and `MODEL_PROVIDER_MAPPING` in [`prompt.py`](https://github.com/interviewstreet/hiring-agent/blob/main/prompt.py) - **Configure** environment variables for authentication and load them in the configuration module - **Extend** `initialize_llm_provider` in [`llm_utils.py`](https://github.com/interviewstreet/hiring-agent/blob/main/llm_utils.py) to handle the new enum case and instantiate your class - **Test** thoroughly to ensure fallback logic and response formatting work as expected across all consumers ## Frequently Asked Questions ### What is the LLMProvider protocol in the hiring agent? The `LLMProvider` protocol is an interface defined in [`models.py`](https://github.com/interviewstreet/hiring-agent/blob/main/models.py) that requires implementing classes to provide a `chat(self, model, messages, options=None, **kwargs) -> Dict` method. This standardization allows the hiring agent to switch between Ollama, Gemini, or custom providers without changing calling code in [`evaluator.py`](https://github.com/interviewstreet/hiring-agent/blob/main/evaluator.py) or [`github.py`](https://github.com/interviewstreet/hiring-agent/blob/main/github.py), as all providers return a uniform response structure. ### How does the hiring agent select which LLM provider to use? The hiring agent selects providers at runtime via the `initialize_llm_provider` function in [`llm_utils.py`](https://github.com/interviewstreet/hiring-agent/blob/main/llm_utils.py). This factory function looks up the requested model name in `MODEL_PROVIDER_MAPPING` (defined in [`prompt.py`](https://github.com/interviewstreet/hiring-agent/blob/main/prompt.py)), instantiates the corresponding provider class (e.g., `OpenAIProvider`), and returns it to the caller. If credentials are missing or the model is unrecognized, it automatically falls back to `OllamaProvider`. ### Can I integrate Anthropic or Cohere using this same process? Yes. Any LLM service that offers a Python SDK or HTTP API can be integrated by following the same seven steps: create a provider class implementing the `chat` method, add an enum value to `ModelProvider`, map your model names in `MODEL_PROVIDER_MAPPING`, expose API keys via environment variables, and extend `initialize_llm_provider` to handle the new enum case with appropriate authentication checks. ### What happens if the API key for my new provider is missing? The `initialize_llm_provider` function includes defensive logic that checks for the presence of required environment variables (e.g., `OPENAI_API_KEY`). If the variable is missing or empty, the function logs a warning message ("⚠️ OpenAI API key missing – falling back to Ollama") and returns an `OllamaProvider` instance instead, ensuring the hiring agent remains functional using local models without crashing.