How to Add a New LLM Provider (Ollama, Gemini, or Custom) to the Hiring Agent Pipeline

To add a new LLM provider to the hiring-agent pipeline, implement the LLMProvider protocol in models.py, register the model-to-provider mapping in prompt.py, and extend the initialize_llm_provider function in llm_utils.py to instantiate your class based on the model name.

The interviewstreet/hiring-agent repository abstracts LLM access behind a provider interface that selects concrete implementations at runtime. This architecture allows you to integrate services like Ollama, Gemini, or any custom API by implementing three specific integration points without modifying the core evaluation logic in evaluator.py or data extraction modules.

Implement the Provider Class in models.py

Create a provider class that conforms to the LLMProvider protocol by implementing the chat(...) method. According to the source code, the method must accept model, messages, options, and **kwargs, and return a normalized dictionary with the schema {"message": {"role": "assistant", "content": "..."}}.

Use the existing implementations as templates:

• OllamaProvider at lines 71-87 in models.py
• GeminiProvider at lines 122-152 in models.py

# models.py

class MyNewProvider:
    """Custom LLM provider implementation."""
    
    def __init__(self, api_key: str = ""):
        # Import and configure the SDK for the new service

        import mynewapi as newapi
        self.client = newapi.Client(api_key=api_key)

    def chat(
        self,
        model: str,
        messages: List[Dict[str, str]],
        options: Dict[str, Any] = None,
        **kwargs,
    ) -> Dict[str, Any]:
        """Send a chat request and return an Ollama‑compatible dict."""
        # Convert the generic `messages` list to the provider’s request format

        request = self._to_provider_format(messages, options)
        response = self.client.chat(model=model, **request)

        # Normalise to {"message": {"role": "assistant", "content": "..."}}

        return {"message": {"role": "assistant", "content": response.text}}

Extend the ModelProvider Enum (Optional)

If your new service requires a distinct enum value, add it to the ModelProvider enum defined at lines 6-11 in models.py:

# models.py

class ModelProvider(Enum):
    OLLAMA = "ollama"
    GEMINI = "gemini"
    MYNEW = "mynew"          # ← new entry

Register the Model-to-Provider Mapping in prompt.py

Map the model names that should use your new provider in the MODEL_PROVIDER_MAPPING dictionary located at lines 46-64 in prompt.py:

# prompt.py

MODEL_PROVIDER_MAPPING = {
    # … existing entries …

    "mynew‑large": ModelProvider.MYNEW,   # ← new model → new provider

}

If you are adding support for a new model that uses an existing provider (for example, a new Ollama model), map it to ModelProvider.OLLAMA instead of creating a new enum value.

Update the Provider Initializer in llm_utils.py

Extend the initialize_llm_provider function at lines 40-63 in llm_utils.py to instantiate your class when the mapped model is requested. The function reads MODEL_PROVIDER_MAPPING and falls back to OllamaProvider by default.

# llm_utils.py

def initialize_llm_provider(model_name: str) -> Any:
    provider = OllamaProvider()                     # default

    model_provider = MODEL_PROVIDER_MAPPING.get(
        model_name, ModelProvider.OLLAMA
    )
    if model_provider == ModelProvider.GEMINI:
        # existing Gemini handling …

    elif model_provider == ModelProvider.MYNEW:    # ← new branch

        provider = MyNewProvider(api_key=os.getenv("MYNEW_API_KEY", ""))
    # else keep default Ollama

    return provider

Configure Environment Variables

If the new SDK requires an API key, expose it via .env.example and read it in the provider’s __init__ method. Follow the existing pattern shown in prompt.py at lines 66-68 and in GeminiProvider at lines 16-20 in models.py.

# .env.example

MYNEW_API_KEY=your_api_key_here

Verify Integration Across the Pipeline

The provider is consumed automatically wherever the pipeline calls initialize_llm_provider. You do not need to modify these call sites:

• Evaluation engine: evaluator.py at lines 34-38
• PDF generation: pdf.py at lines 23 and 41-45
• GitHub data extraction: github.py at lines 370-372

Run the pipeline with your new model name to verify:

export DEFAULT_MODEL="mynew-large"
python -m main.evaluator

The console output will indicate which provider is selected via the logger.info statements inside initialize_llm_provider.

Summary

• Implement the protocol: Create a class in models.py with a chat() method that returns {"message": {"role": "assistant", "content": "..."}}
• Map the model: Add entries to MODEL_PROVIDER_MAPPING in prompt.py to associate model names with your provider
• Wire the initializer: Add an elif branch in initialize_llm_provider ( llm_utils.py lines 40-63) to instantiate your class
• Configure secrets: Add API keys to .env.example and read them via os.getenv() in your provider's constructor
• Test transparently: The rest of the codebase in evaluator.py, pdf.py, and github.py will use your provider automatically when the mapped model name is requested

Frequently Asked Questions

What interface must a new LLM provider implement?

A new provider must implement the chat(self, model, messages, options=None, **kwargs) method and return a dictionary formatted as {"message": {"role": "assistant", "content": "..."}}. This normalization allows the rest of the hiring-agent pipeline to treat all providers interchangeably, as seen in the OllamaProvider and GeminiProvider implementations in models.py.

Where is the mapping between model names and providers defined?

The mapping is defined in the MODEL_PROVIDER_MAPPING dictionary in prompt.py at lines 46-64. This dictionary maps string model names (like "gemini-pro" or "llama2") to ModelProvider enum values, which initialize_llm_provider in llm_utils.py uses to select the correct implementation.

Do I need to modify the evaluation logic to use a new provider?

No. The evaluation logic in evaluator.py (lines 34-38), PDF generation in pdf.py (lines 41-45), and GitHub extraction in github.py (lines 370-372) all call initialize_llm_provider(DEFAULT_MODEL). As long as you register your provider in the mapping and initializer, these modules will use your new provider transparently without code changes.

How do I handle API keys for a custom provider?

Store the API key in your .env file (add it to .env.example for documentation), then read it inside your provider's __init__ method using os.getenv(), following the pattern used by GeminiProvider at lines 16-20 in models.py and the environment handling at lines 66-68 in prompt.py.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}