How to Set the LLM Provider in Hiring Agent: Ollama vs Gemini Configuration

Set the LLM_PROVIDER environment variable to ollama or gemini in your .env file, and ensure GEMINI_API_KEY is present when using Gemini; the initialize_llm_provider function in llm_utils.py handles the runtime instantiation based on the MODEL_PROVIDER_MAPPING table.

The Hiring Agent open-source repository (interviewstreet/hiring-agent) supports multiple Large Language Model (LLM) backends, allowing you to choose between local inference via Ollama or cloud-based generation through Google Gemini. Understanding how to configure and switch between these providers ensures your hiring automation pipeline uses the right model for your infrastructure and privacy requirements. This guide explains the exact configuration mechanism, file locations, and initialization patterns used in the source code.

Understanding the Provider Selection Architecture

The Hiring Agent uses a provider-agnostic architecture that decouples the LLM implementation from the business logic. The system determines which backend to use through a combination of environment variables and a static mapping table defined in the source code.

The ModelProvider Enum (models.py)

At the core of the selection logic sits the ModelProvider enum defined in [models.py](https://github.com/interviewstreet/hiring-agent/blob/main/models.py). This enum defines two possible values:

• OLLAMA – Represents the local Ollama server provider
• GEMINI – Represents the Google Gemini API provider

Additionally, models.py declares the LLMProvider protocol, which both concrete providers implement. This protocol ensures that regardless of which backend is active, calling code can invoke the unified chat method with consistent parameters.

Environment Variable Defaults (prompt.py)

Default values and environment variable handling reside in [prompt.py](https://github.com/interviewstreet/hiring-agent/blob/main/prompt.py). This file reads the LLM_PROVIDER variable (defaulting to "ollama" if unset) and checks for the presence of GEMINI_API_KEY. It also contains the MODEL_PROVIDER_MAPPING dictionary, which maps specific model strings (e.g., "gemma3:4b", "gemini-2.5-pro") to their corresponding ModelProvider enum values.

Configuring Ollama vs Gemini

Switching between providers requires specific environment configurations. The system validates credentials at runtime and applies fallback logic to prevent startup failures.

Local Ollama Setup

To use Ollama for local inference:

1. Set LLM_PROVIDER=ollama in your environment
2. Ensure your Ollama server is running and accessible
3. Optionally specify a local model via DEFAULT_MODEL (e.g., gemma3:4b)

No API key is required for Ollama operation. If you omit LLM_PROVIDER, the system defaults to Ollama automatically.

Google Gemini API Setup

To configure Gemini for cloud-based generation:

1. Set LLM_PROVIDER=gemini in your .env file or shell environment
2. Export a valid GEMINI_API_KEY with appropriate permissions for the Gemini API
3. Set DEFAULT_MODEL to a Gemini-specific identifier (e.g., gemini-2.5-pro)

If you specify Gemini as the provider but fail to provide GEMINI_API_KEY, the code logs a warning and gracefully falls back to Ollama rather than crashing.

Runtime Provider Initialization

The [llm_utils.py](https://github.com/interviewstreet/hiring-agent/blob/main/llm_utils.py) file contains the critical initialize_llm_provider(model_name) function. This function:

• Accepts a model name (e.g., "gemini-2.5-pro")
• Looks up the provider type in MODEL_PROVIDER_MAPPING
• Instantiates either OllamaProvider or GeminiProvider based on the mapping and environment variables
• Returns an object conforming to the LLMProvider protocol

All higher-level application code calls this factory function rather than instantiating providers directly, ensuring consistent behavior across the codebase.

Practical Configuration Examples

Environment File Configuration

Create a .env file in your project root to configure the provider:

# .env

LLM_PROVIDER=gemini          # Use "ollama" for local inference

GEMINI_API_KEY=YOUR_KEY_HERE # Required only for Gemini

DEFAULT_MODEL=gemini-2.5-pro # Optional: specify target model

Programmatic Provider Initialization

You can manually initialize and use the provider in Python:

from llm_utils import initialize_llm_provider

# Select a model defined in MODEL_PROVIDER_MAPPING

model = "gemini-2.5-pro"

# Returns either OllamaProvider or GeminiProvider instance

llm = initialize_llm_provider(model)

# Call the unified interface

response = llm.chat(
    model=model,
    messages=[{"role": "user", "content": "Explain the hiring process."}],
    options={}  # Model-specific parameters from MODEL_PARAMETERS

)
print(response["message"]["content"])

Verifying Active Configuration

To confirm which provider is currently active:

import os
from prompt import PROVIDER

print(f"The current LLM provider is: {PROVIDER}")

# Outputs: "ollama" or "gemini"

Summary

• LLM_PROVIDER environment variable controls which backend the Hiring Agent uses at runtime.
• prompt.py defines the default values and MODEL_PROVIDER_MAPPING that associates model names with provider types.
• initialize_llm_provider in llm_utils.py is the factory function that creates the appropriate provider instance based on your configuration.
• Gemini requires GEMINI_API_KEY; if missing, the system falls back to Ollama.
• Ollama requires no API key and serves as the default when no configuration is provided.

Frequently Asked Questions

How do I switch from Ollama to Gemini without modifying code?

Set LLM_PROVIDER=gemini in your environment or .env file and ensure GEMINI_API_KEY contains a valid Google API key. The initialize_llm_provider function automatically detects this change on the next application start and instantiates the GeminiProvider instead of OllamaProvider.

What happens if I forget to set GEMINI_API_KEY when using Gemini?

The code in prompt.py detects the missing credential, logs a warning message, and falls back to the Ollama provider. This ensures the Hiring Agent remains functional even when API keys are temporarily unavailable, defaulting to local inference.

Where is the list of supported models for each provider?

The supported models are defined in the MODEL_PROVIDER_MAPPING dictionary located in [prompt.py](https://github.com/interviewstreet/hiring-agent/blob/main/prompt.py). This mapping connects specific model strings like "gemma3:4b" or "gemini-2.5-pro" to the OLLAMA or GEMINI enum values from [models.py](https://github.com/interviewstreet/hiring-agent/blob/main/models.py).

Can I use both providers simultaneously in the same application instance?

While the environment variable determines the default provider, you can instantiate specific providers manually by importing the classes directly from llm_utils.py. However, the standard initialize_llm_provider factory function returns a single configured instance based on the global LLM_PROVIDER setting.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}