# How to Integrate a New LLM Provider into the Hiring Agent System > Learn how to integrate a new LLM provider into the hiring agent system. This guide covers subclassing BaseLLM, registering your class, and configuring credentials for seamless integration. - Repository: [HackerRank/hiring-agent](https://github.com/interviewstreet/hiring-agent) - Tags: how-to-guide - Published: 2026-06-27 --- **Integrating a new LLM provider into the interviewstreet/hiring-agent system requires subclassing `BaseLLM` in [`models.py`](https://github.com/interviewstreet/hiring-agent/blob/main/models.py), registering the class in `MODEL_REGISTRY`, and adding provider credentials to [`config.py`](https://github.com/interviewstreet/hiring-agent/blob/main/config.py), enabling automatic discovery by the high-level utilities in [`llm_utils.py`](https://github.com/interviewstreet/hiring-agent/blob/main/llm_utils.py).** The interviewstreet/hiring-agent repository provides a modular hiring agent system built on a provider-agnostic abstraction layer. To integrate a new LLM provider, you work with the registry-based architecture defined in [`models.py`](https://github.com/interviewstreet/hiring-agent/blob/main/models.py) and [`config.py`](https://github.com/interviewstreet/hiring-agent/blob/main/config.py), ensuring your implementation conforms to the unified interface expected by the rest of the codebase. This design allows you to add support for additional large language models without modifying evaluation logic or prompt engineering code. ## Understanding the LLM Abstraction Architecture The hiring agent system isolates provider-specific logic behind a thin abstraction layer. According to the source code structure, all LLM interactions flow through three core components: - **[`models.py`](https://github.com/interviewstreet/hiring-agent/blob/main/models.py)** – Contains the abstract `BaseLLM` class and the `MODEL_REGISTRY` dictionary that maps provider names to concrete implementations. - **[`config.py`](https://github.com/interviewstreet/hiring-agent/blob/main/config.py)** – Defines the `LLM_PROVIDERS` configuration schema and validates environment variables for API keys. - **[`llm_utils.py`](https://github.com/interviewstreet/hiring-agent/blob/main/llm_utils.py)** – Provides high-level wrappers like `generate_text()` and `stream_chat()` that consume the registry to route requests to the appropriate provider. This architecture means the evaluation engine in [`evaluator.py`](https://github.com/interviewstreet/hiring-agent/blob/main/evaluator.py) and prompt builders in [`prompt.py`](https://github.com/interviewstreet/hiring-agent/blob/main/prompt.py) remain completely decoupled from specific LLM implementations. They call only the abstract interface, making provider integration a matter of extending the registry rather than refactoring business logic. ## Step-by-Step Integration Guide Follow these steps to add a new LLM provider to the hiring agent system. Each step targets a specific file in the codebase to ensure clean, modular integration. ### Step 1: Configure Provider Credentials in [`config.py`](https://github.com/interviewstreet/hiring-agent/blob/main/config.py) Add your provider to the `LLM_PROVIDERS` dictionary in [`config.py`](https://github.com/interviewstreet/hiring-agent/blob/main/config.py). This configuration object maps the provider identifier to its required environment variable names. ```python # config.py LLM_PROVIDERS = { "openai": {"api_key_env": "OPENAI_API_KEY"}, "anthropic": {"api_key_env": "ANTHROPIC_API_KEY"}, # New provider entry "myllm": {"api_key_env": "MYLLM_API_KEY"}, } ``` Update `.env.example` to document the new environment variable so other developers know what credentials to provide: ```bash # .env.example MYLLM_API_KEY=your-myllm-api-key-here ``` ### Step 2: Implement the Model Class in [`models.py`](https://github.com/interviewstreet/hiring-agent/blob/main/models.py) Create a new subclass of `BaseLLM` in [`models.py`](https://github.com/interviewstreet/hiring-agent/blob/main/models.py). You must implement at least the `chat(messages, **kwargs)` and `completion(prompt, **kwargs)` methods to satisfy the interface contract. ```python # models.py import os import requests from typing import List, Dict, Any class MyLLM(BaseLLM): """ Concrete implementation for the MyLLM API. """ endpoint = "https://api.myllm.com/v1/chat" def __init__(self): self.api_key = os.getenv("MYLLM_API_KEY") if not self.api_key: raise ValueError("MYLLM_API_KEY environment variable not set") def _post(self, payload: dict) -> dict: headers = {"Authorization": f"Bearer {self.api_key}"} response = requests.post( self.endpoint, json=payload, headers=headers, timeout=30 ) response.raise_for_status() return response.json() def chat(self, messages: List[Dict[str, str]], **kwargs) -> str: """ messages format: [{"role": "user", "content": "..."}, ...] Returns the generated response string. """ payload = {"messages": messages, **kwargs} data = self._post(payload) return data["choices"][0]["message"] def completion(self, prompt: str, **kwargs) -> str: """ For providers supporting direct prompt completion. """ payload = {"prompt": prompt, **kwargs} data = self._post(payload) return data["choices"][0]["text"] ``` ### Step 3: Register the Provider in `MODEL_REGISTRY` Add your class to the registry dictionary near the bottom of [`models.py`](https://github.com/interviewstreet/hiring-agent/blob/main/models.py). The key must match the provider identifier used in [`config.py`](https://github.com/interviewstreet/hiring-agent/blob/main/config.py). ```python # models.py MODEL_REGISTRY = { "openai": OpenAIModel, "anthropic": AnthropicModel, "myllm": MyLLM, # New provider registration } ``` ### Step 4: Verify Integration Through [`llm_utils.py`](https://github.com/interviewstreet/hiring-agent/blob/main/llm_utils.py) The high-level utilities in [`llm_utils.py`](https://github.com/interviewstreet/hiring-agent/blob/main/llm_utils.py) automatically resolve the correct provider class based on the configuration. Test your integration by calling the utility functions: ```python # Usage example (not a file in the repo) from llm_utils import generate_text # This will instantiate MyLLM and route the request result = generate_text( provider="myllm", prompt="Evaluate this candidate's response", max_tokens=512 ) ``` ## Complete Implementation Example Here is the minimal viable implementation for a new provider called "MyLLM" that follows the exact patterns found in the interviewstreet/hiring-agent source code: **File: [`models.py`](https://github.com/interviewstreet/hiring-agent/blob/main/models.py)** ```python import os import requests from abc import ABC, abstractmethod class BaseLLM(ABC): """Abstract base class existing in the repo""" @abstractmethod def chat(self, messages, **kwargs): pass @abstractmethod def completion(self, prompt, **kwargs): pass class MyLLM(BaseLLM): endpoint = "https://api.myllm.com/v1/chat" def __init__(self): config = get_config() # From config.py self.api_key = os.getenv(config["myllm"]["api_key_env"]) def _post(self, payload): headers = {"Authorization": f"Bearer {self.api_key}"} resp = requests.post(self.endpoint, json=payload, headers=headers) return resp.json() def chat(self, messages, **kwargs): payload = {"messages": messages, "max_tokens": kwargs.get("max_tokens", 512)} return self._post(payload)["choices"][0]["message"] def completion(self, prompt, **kwargs): payload = {"prompt": prompt, "max_tokens": kwargs.get("max_tokens", 512)} return self._post(payload)["choices"][0]["text"] # Registration MODEL_REGISTRY = { "openai": OpenAIModel, "anthropic": AnthropicModel, "myllm": MyLLM, } ``` **File: [`config.py`](https://github.com/interviewstreet/hiring-agent/blob/main/config.py)** ```python LLM_PROVIDERS = { "openai": {"api_key_env": "OPENAI_API_KEY"}, "anthropic": {"api_key_env": "ANTHROPIC_API_KEY"}, "myllm": {"api_key_env": "MYLLM_API_KEY"}, } ``` ## Verifying Your Integration After implementing the new provider, validate the integration using the existing test suite. The repository's unit tests target the abstract `BaseLLM` interface, meaning they will automatically exercise your new implementation once configured with valid credentials. Run the test suite with: ```bash export MYLLM_API_KEY="your-test-key" pytest tests/test_llm_utils.py -v ``` Check that [`llm_utils.py`](https://github.com/interviewstreet/hiring-agent/blob/main/llm_utils.py) correctly instantiates your class when the provider configuration points to your new implementation. The utilities should handle `MyLLM` identically to built-in providers like OpenAI or Anthropic. ## Summary - **Registry Pattern**: The hiring agent system uses a `MODEL_REGISTRY` in [`models.py`](https://github.com/interviewstreet/hiring-agent/blob/main/models.py) to map provider names to concrete implementations, decoupling business logic from LLM specifics. - **Four File Changes**: Integration requires editing only [`config.py`](https://github.com/interviewstreet/hiring-agent/blob/main/config.py) (credentials), [`models.py`](https://github.com/interviewstreet/hiring-agent/blob/main/models.py) (implementation and registration), and `.env.example` (documentation). - **Interface Contract**: New providers must subclass `BaseLLM` and implement `chat()` and `completion()` methods to ensure compatibility with [`llm_utils.py`](https://github.com/interviewstreet/hiring-agent/blob/main/llm_utils.py). - **Zero Refactoring**: Because [`evaluator.py`](https://github.com/interviewstreet/hiring-agent/blob/main/evaluator.py) and [`prompt.py`](https://github.com/interviewstreet/hiring-agent/blob/main/prompt.py) depend only on the abstract interface, no other files require modification when adding a new LLM provider. ## Frequently Asked Questions ### What is the minimum required method implementation for a new LLM provider? You must implement `chat(messages, **kwargs)` and `completion(prompt, **kwargs)` in your `BaseLLM` subclass. The `chat` method accepts a list of message dictionaries (following the OpenAI format with `role` and `content` keys) and returns a string response. The `completion` method accepts a single prompt string and returns the generated text. These methods enable full compatibility with the [`llm_utils.py`](https://github.com/interviewstreet/hiring-agent/blob/main/llm_utils.py) wrappers. ### Where does the hiring agent system store provider API keys? The system stores API keys in environment variables referenced by the `LLM_PROVIDERS` dictionary in [`config.py`](https://github.com/interviewstreet/hiring-agent/blob/main/config.py). Each provider entry maps to an `api_key_env` string that specifies which environment variable contains the credential. For example, OpenAI uses `OPENAI_API_KEY`, while your new provider might use `MYLLM_API_KEY`. The concrete model class reads this value during initialization. ### Do I need to modify the evaluation logic when adding a new provider? No. The evaluation logic in [`evaluator.py`](https://github.com/interviewstreet/hiring-agent/blob/main/evaluator.py) and prompt construction in [`prompt.py`](https://github.com/interviewstreet/hiring-agent/blob/main/prompt.py) interact exclusively with the high-level functions in [`llm_utils.py`](https://github.com/interviewstreet/hiring-agent/blob/main/llm_utils.py) or the abstract `BaseLLM` interface. Because your new provider implements this standard interface and registers in `MODEL_REGISTRY`, existing evaluation workflows will automatically route to your new LLM without code changes. ### How does the system handle streaming responses for new providers? If your LLM supports streaming, implement a `stream_chat()` method in your `BaseLLM` subclass that yields response chunks incrementally. The [`llm_utils.py`](https://github.com/interviewstreet/hiring-agent/blob/main/llm_utils.py) module detects the presence of this method and routes streaming requests accordingly. If you only implement the standard `chat()` method, the system will fall back to synchronous responses automatically.