How to Switch LLM Providers in Hiring Agent: Ollama vs Gemini Configuration

Hiring Agent selects between local Ollama and Google Gemini at runtime via the LLM_PROVIDER environment variable, automatically instantiating the correct provider class through the initialize_llm_provider function in llm_utils.py.

The open-source Hiring Agent repository (interviewstreet/hiring-agent) provides a flexible abstraction layer that allows you to switch LLM providers without changing application code. Whether you need to run models locally for privacy or leverage Google's Gemini API for advanced reasoning, the system uses a provider pattern to handle both scenarios through a unified interface. This guide explains how to configure and switch between Ollama and Gemini providers using environment variables and the internal mapping system.

Understanding the Provider Selection Logic

Hiring Agent determines which LLM service to use by reading configuration values and consulting a mapping table defined in the source code.

Environment Variable Configuration

The primary mechanism for switching providers is the LLM_PROVIDER environment variable, defined alongside defaults in [prompt.py](https://github.com/interviewstreet/hiring-agent/blob/main/prompt.py). The system recognizes two valid values:

• ollama – Routes requests to a local Ollama server using the OllamaProvider class.
• gemini – Routes requests to the Google Gemini API using the GeminiProvider class, provided a valid GEMINI_API_KEY is present.

For Gemini users, you must also set GEMINI_API_KEY with your Google AI Studio API key. Ollama users do not require an API key since inference happens locally.

Model-to-Provider Mapping

Each supported model name maps to a specific provider through the MODEL_PROVIDER_MAPPING dictionary in prompt.py. This map associates model strings like gemma3:4b or gemini-2.5-pro with the ModelProvider enum defined in [models.py](https://github.com/interviewstreet/hiring-agent/blob/main/models.py). The enum contains two values: OLLAMA and GEMINI.

When you request a model, the system checks this mapping to validate that the chosen model belongs to the configured provider.

Configuring Ollama as Your LLM Provider

To use a local Ollama instance for private, on-premise inference, set your environment as follows:

# .env

LLM_PROVIDER=ollama
DEFAULT_MODEL=gemma3:4b

# GEMINI_API_KEY can be omitted or left empty

In this configuration, Hiring Agent instantiates OllamaProvider, which communicates with your local Ollama server (typically running on localhost:11434). Ensure your desired model (e.g., gemma3:4b) is pulled and available in your local Ollama instance before running the application.

Configuring Google Gemini as Your LLM Provider

To switch to Google's hosted models, update your environment variables to trigger the GeminiProvider:

# .env

LLM_PROVIDER=gemini
GEMINI_API_KEY=your_google_ai_studio_key_here
DEFAULT_MODEL=gemini-2.5-pro

The application validates that GEMINI_API_KEY exists before attempting to create a GeminiProvider instance. If the key is missing, the system logs a warning and falls back to Ollama to prevent runtime crashes.

Runtime Provider Initialization

All higher-level code interacts with LLMs through the initialize_llm_provider function located in [llm_utils.py](https://github.com/interviewstreet/hiring-agent/blob/main/llm_utils.py). This factory function reads the model_name, checks it against MODEL_PROVIDER_MAPPING, and returns an object implementing the LLMProvider protocol (defined in models.py).

Here is how to manually initialize and use the provider in Python:

from llm_utils import initialize_llm_provider

# Specify a model listed in MODEL_PROVIDER_MAPPING

model = "gemini-2.5-pro"

# Returns either OllamaProvider or GeminiProvider based on LLM_PROVIDER

llm = initialize_llm_provider(model)

# The unified chat interface works identically for both providers

response = llm.chat(
    model=model,
    messages=[{"role": "user", "content": "Analyze this candidate's resume."}],
    options={},  # Model-specific parameters from MODEL_PARAMETERS

)
print(response["message"]["content"])

Both provider classes implement the same chat method signature, ensuring that switching providers requires no changes to your business logic—only environment configuration.

Fallback Behavior and Error Handling

Hiring Agent includes defensive logic to prevent startup failures due to missing credentials. If you set LLM_PROVIDER=gemini but fail to provide a GEMINI_API_KEY, the application emits a warning log and automatically falls back to OllamaProvider. This ensures the hiring pipeline remains operational even when API keys are temporarily unavailable.

To verify which provider is currently active at runtime, inspect the PROVIDER variable exported from prompt.py:

import os
from prompt import PROVIDER

print(f"Active LLM provider: {PROVIDER}")

# Outputs: "ollama" or "gemini"

Summary

• Set LLM_PROVIDER to ollama or gemini in your environment to switch between local and hosted inference.
• Provide GEMINI_API_KEY only when using Gemini; omit it for Ollama.
• The initialize_llm_provider function in llm_utils.py handles instantiation, returning a provider that conforms to the LLMProvider protocol.
• MODEL_PROVIDER_MAPPING in prompt.py links specific model names (e.g., gemma3:4b, gemini-2.5-pro) to their respective providers.
• Missing Gemini credentials trigger a fallback to Ollama, guaranteeing application stability.

Frequently Asked Questions

What happens if I set LLM_PROVIDER to gemini without providing an API key?

According to the source code in prompt.py and llm_utils.py, the application detects the missing GEMINI_API_KEY, logs a warning, and falls back to using OllamaProvider to ensure the service continues running. You should check your logs if you notice unexpected local inference when Gemini was intended.

Where is the provider selection logic implemented?

The selection logic resides in two critical files: prompt.py defines the LLM_PROVIDER default and the MODEL_PROVIDER_MAPPING table, while llm_utils.py contains the initialize_llm_provider function that performs the runtime instantiation of OllamaProvider or GeminiProvider based on the environment configuration.

Can I use other models besides gemma3:4b and gemini-2.5-pro?

Yes, provided you update the MODEL_PROVIDER_MAPPING dictionary in prompt.py to include your custom model strings mapped to the appropriate ModelProvider.OLLAMA or ModelProvider.GEMINI enum value. The initialize_llm_provider function validates model names against this mapping before creating the provider instance.

How do I verify which provider is currently active?

Import the PROVIDER constant from prompt.py and print its value, or check your environment variables before starting the application. The PROVIDER variable reflects the final resolved value of LLM_PROVIDER and indicates whether OllamaProvider or GeminiProvider is handling your requests.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}