How to Integrate a New LLM Provider into the Hiring Agent System

Integrating a new LLM provider into the interviewstreet/hiring-agent system requires subclassing BaseLLM in models.py, registering the class in MODEL_REGISTRY, and adding provider credentials to config.py, enabling automatic discovery by the high-level utilities in llm_utils.py.

The interviewstreet/hiring-agent repository provides a modular hiring agent system built on a provider-agnostic abstraction layer. To integrate a new LLM provider, you work with the registry-based architecture defined in models.py and config.py, ensuring your implementation conforms to the unified interface expected by the rest of the codebase. This design allows you to add support for additional large language models without modifying evaluation logic or prompt engineering code.

Understanding the LLM Abstraction Architecture

The hiring agent system isolates provider-specific logic behind a thin abstraction layer. According to the source code structure, all LLM interactions flow through three core components:

• models.py – Contains the abstract BaseLLM class and the MODEL_REGISTRY dictionary that maps provider names to concrete implementations.
• config.py – Defines the LLM_PROVIDERS configuration schema and validates environment variables for API keys.
• llm_utils.py – Provides high-level wrappers like generate_text() and stream_chat() that consume the registry to route requests to the appropriate provider.

This architecture means the evaluation engine in evaluator.py and prompt builders in prompt.py remain completely decoupled from specific LLM implementations. They call only the abstract interface, making provider integration a matter of extending the registry rather than refactoring business logic.

Step-by-Step Integration Guide

Follow these steps to add a new LLM provider to the hiring agent system. Each step targets a specific file in the codebase to ensure clean, modular integration.

Step 1: Configure Provider Credentials in config.py

Add your provider to the LLM_PROVIDERS dictionary in config.py. This configuration object maps the provider identifier to its required environment variable names.

# config.py

LLM_PROVIDERS = {
    "openai": {"api_key_env": "OPENAI_API_KEY"},
    "anthropic": {"api_key_env": "ANTHROPIC_API_KEY"},
    # New provider entry

    "myllm": {"api_key_env": "MYLLM_API_KEY"},
}

Update .env.example to document the new environment variable so other developers know what credentials to provide:

# .env.example

MYLLM_API_KEY=your-myllm-api-key-here

Step 2: Implement the Model Class in models.py

Create a new subclass of BaseLLM in models.py. You must implement at least the chat(messages, **kwargs) and completion(prompt, **kwargs) methods to satisfy the interface contract.

# models.py

import os
import requests
from typing import List, Dict, Any

class MyLLM(BaseLLM):
    """
    Concrete implementation for the MyLLM API.
    """
    endpoint = "https://api.myllm.com/v1/chat"
    
    def __init__(self):
        self.api_key = os.getenv("MYLLM_API_KEY")
        if not self.api_key:
            raise ValueError("MYLLM_API_KEY environment variable not set")

    def _post(self, payload: dict) -> dict:
        headers = {"Authorization": f"Bearer {self.api_key}"}
        response = requests.post(
            self.endpoint, 
            json=payload, 
            headers=headers, 
            timeout=30
        )
        response.raise_for_status()
        return response.json()

    def chat(self, messages: List[Dict[str, str]], **kwargs) -> str:
        """
        messages format: [{"role": "user", "content": "..."}, ...]
        Returns the generated response string.
        """
        payload = {"messages": messages, **kwargs}
        data = self._post(payload)
        return data["choices"][0]["message"]

    def completion(self, prompt: str, **kwargs) -> str:
        """
        For providers supporting direct prompt completion.
        """
        payload = {"prompt": prompt, **kwargs}
        data = self._post(payload)
        return data["choices"][0]["text"]

Step 3: Register the Provider in MODEL_REGISTRY

Add your class to the registry dictionary near the bottom of models.py. The key must match the provider identifier used in config.py.

# models.py

MODEL_REGISTRY = {
    "openai": OpenAIModel,
    "anthropic": AnthropicModel,
    "myllm": MyLLM,  # New provider registration

}

Step 4: Verify Integration Through llm_utils.py

The high-level utilities in llm_utils.py automatically resolve the correct provider class based on the configuration. Test your integration by calling the utility functions:

# Usage example (not a file in the repo)

from llm_utils import generate_text

# This will instantiate MyLLM and route the request

result = generate_text(
    provider="myllm",
    prompt="Evaluate this candidate's response",
    max_tokens=512
)

Complete Implementation Example

Here is the minimal viable implementation for a new provider called "MyLLM" that follows the exact patterns found in the interviewstreet/hiring-agent source code:

File: models.py

import os
import requests
from abc import ABC, abstractmethod

class BaseLLM(ABC):
    """Abstract base class existing in the repo"""
    @abstractmethod
    def chat(self, messages, **kwargs):
        pass
    
    @abstractmethod
    def completion(self, prompt, **kwargs):
        pass

class MyLLM(BaseLLM):
    endpoint = "https://api.myllm.com/v1/chat"
    
    def __init__(self):
        config = get_config()  # From config.py

        self.api_key = os.getenv(config["myllm"]["api_key_env"])
    
    def _post(self, payload):
        headers = {"Authorization": f"Bearer {self.api_key}"}
        resp = requests.post(self.endpoint, json=payload, headers=headers)
        return resp.json()
    
    def chat(self, messages, **kwargs):
        payload = {"messages": messages, "max_tokens": kwargs.get("max_tokens", 512)}
        return self._post(payload)["choices"][0]["message"]
    
    def completion(self, prompt, **kwargs):
        payload = {"prompt": prompt, "max_tokens": kwargs.get("max_tokens", 512)}
        return self._post(payload)["choices"][0]["text"]

# Registration

MODEL_REGISTRY = {
    "openai": OpenAIModel,
    "anthropic": AnthropicModel,
    "myllm": MyLLM,
}

File: config.py

LLM_PROVIDERS = {
    "openai": {"api_key_env": "OPENAI_API_KEY"},
    "anthropic": {"api_key_env": "ANTHROPIC_API_KEY"},
    "myllm": {"api_key_env": "MYLLM_API_KEY"},
}

Verifying Your Integration

After implementing the new provider, validate the integration using the existing test suite. The repository's unit tests target the abstract BaseLLM interface, meaning they will automatically exercise your new implementation once configured with valid credentials.

Run the test suite with:

export MYLLM_API_KEY="your-test-key"
pytest tests/test_llm_utils.py -v

Check that llm_utils.py correctly instantiates your class when the provider configuration points to your new implementation. The utilities should handle MyLLM identically to built-in providers like OpenAI or Anthropic.

Summary

• Registry Pattern: The hiring agent system uses a MODEL_REGISTRY in models.py to map provider names to concrete implementations, decoupling business logic from LLM specifics.
• Four File Changes: Integration requires editing only config.py (credentials), models.py (implementation and registration), and .env.example (documentation).
• Interface Contract: New providers must subclass BaseLLM and implement chat() and completion() methods to ensure compatibility with llm_utils.py.
• Zero Refactoring: Because evaluator.py and prompt.py depend only on the abstract interface, no other files require modification when adding a new LLM provider.

Frequently Asked Questions

What is the minimum required method implementation for a new LLM provider?

You must implement chat(messages, **kwargs) and completion(prompt, **kwargs) in your BaseLLM subclass. The chat method accepts a list of message dictionaries (following the OpenAI format with role and content keys) and returns a string response. The completion method accepts a single prompt string and returns the generated text. These methods enable full compatibility with the llm_utils.py wrappers.

Where does the hiring agent system store provider API keys?

The system stores API keys in environment variables referenced by the LLM_PROVIDERS dictionary in config.py. Each provider entry maps to an api_key_env string that specifies which environment variable contains the credential. For example, OpenAI uses OPENAI_API_KEY, while your new provider might use MYLLM_API_KEY. The concrete model class reads this value during initialization.

Do I need to modify the evaluation logic when adding a new provider?

No. The evaluation logic in evaluator.py and prompt construction in prompt.py interact exclusively with the high-level functions in llm_utils.py or the abstract BaseLLM interface. Because your new provider implements this standard interface and registers in MODEL_REGISTRY, existing evaluation workflows will automatically route to your new LLM without code changes.

How does the system handle streaming responses for new providers?

If your LLM supports streaming, implement a stream_chat() method in your BaseLLM subclass that yields response chunks incrementally. The llm_utils.py module detects the presence of this method and routes streaming requests accordingly. If you only implement the standard chat() method, the system will fall back to synchronous responses automatically.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}