How to Configure Hiring Agent for Ollama vs Google Gemini: Runtime Provider Setup

Configure the interviewstreet/hiring-agent to use Ollama or Google Gemini by setting the LLM_PROVIDER environment variable and ensuring the GEMINI_API_KEY is present for cloud inference, or simply pass a Gemini model name to the ResumeEvaluator class to trigger automatic provider detection via the MODEL_PROVIDER_MAPPING table.

The hiring-agent repository supports both local LLM hosting through Ollama and managed cloud inference via Google Gemini. Provider selection happens at runtime through the initialize_llm_provider function in llm_utils.py, which inspects environment variables and model name mappings to instantiate either OllamaProvider or GeminiProvider.

How Provider Selection Works

The initialize_llm_provider function in [llm_utils.py](https://github.com/interviewstreet/hiring-agent/blob/main/llm_utils.py) implements the core switching logic. When a ResumeEvaluator is instantiated (see [evaluator.py](https://github.com/interviewstreet/hiring-agent/blob/main/evaluator.py), lines 24-40), it calls initialize_llm_provider(self.model_name) to determine which backend to use.

The function applies the following resolution order:

1. Model name lookup – The requested model is checked against MODEL_PROVIDER_MAPPING in [prompt.py](https://github.com/interviewstreet/hiring-agent/blob/main/prompt.py) (lines 48-64). This dictionary maps specific model strings to the ModelProvider.OLLAMA or ModelProvider.GEMINI enum values defined in models.py.
2. API key validation – If the mapping returns GEMINI, the function checks for the GEMINI_API_KEY environment variable (loaded in prompt.py, line 66). If the key is missing, the system logs a warning and falls back to OllamaProvider.
3. Default fallback – If the model is not mapped to Gemini or the provider variable is unset, OllamaProvider is instantiated as the default.

Configuration via Environment Variables

Control the default provider using the LLM_PROVIDER variable and model-specific settings.

For Ollama (Local Inference)

Set the provider to ollama and specify a compatible local model:

# .env file or export statements

LLM_PROVIDER=ollama
DEFAULT_MODEL=gemma3:4b

# GEMINI_API_KEY not required

from evaluator import ResumeEvaluator

evaluator = ResumeEvaluator()  # Uses OllamaProvider automatically

result = evaluator.evaluate_resume(resume_text)

For Google Gemini (Cloud API)

Enable cloud inference by setting the provider to gemini and providing your API key:

# .env file

LLM_PROVIDER=gemini
DEFAULT_MODEL=gemini-3.5-flash
GEMINI_API_KEY=your_secret_key_here

from evaluator import ResumeEvaluator

evaluator = ResumeEvaluator()  # Initializes GeminiProvider

result = evaluator.evaluate_resume(resume_text)

Runtime Provider Override

You can bypass environment variables entirely by passing a specific model name when constructing ResumeEvaluator. The initialize_llm_provider function looks up the model in MODEL_PROVIDER_MAPPING and instantiates the appropriate provider regardless of the LLM_PROVIDER setting.

from evaluator import ResumeEvaluator

# Forces Gemini usage even if LLM_PROVIDER=ollama

evaluator = ResumeEvaluator(model_name="gemini-2.5-pro")
result = evaluator.evaluate_resume(resume_text)

Conversely, passing an Ollama-specific model like gemma3:4b forces local inference even if the environment suggests Gemini.

Key Configuration Files

Understanding these source files helps debug provider selection issues:

• [prompt.py](https://github.com/interviewstreet/hiring-agent/blob/main/prompt.py) – Contains MODEL_PROVIDER_MAPPING (lines 48-64), default model constants, and GEMINI_API_KEY retrieval (line 66).
• [llm_utils.py](https://github.com/interviewstreet/hiring-agent/blob/main/llm_utils.py) – Houses initialize_llm_provider and the concrete OllamaProvider and GeminiProvider implementation classes.
• [evaluator.py](https://github.com/interviewstreet/hiring-agent/blob/main/evaluator.py) – The ResumeEvaluator class calls the initialization logic during construction (lines 24-40).
• [models.py](https://github.com/interviewstreet/hiring-agent/blob/main/models.py) – Defines the ModelProvider enum used throughout the provider selection logic.
• .env.example – Sample environment file demonstrating the required variable format.

Summary

• Environment variable: Set LLM_PROVIDER to ollama or gemini to declare intent, though the model name ultimately determines the instantiated provider.
• Model mapping: The MODEL_PROVIDER_MAPPING table in prompt.py resolves model strings to provider classes.
• API key requirement: GEMINI_API_KEY must be present in the environment when using any Gemini model; absence triggers an automatic fallback to Ollama.
• Runtime flexibility: Pass a specific model_name to ResumeEvaluator to override environment defaults on a per-evaluation basis.

Frequently Asked Questions

What happens if I set LLM_PROVIDER=gemini but forget to set GEMINI_API_KEY?

The initialize_llm_provider function checks for the API key before instantiating GeminiProvider. If GEMINI_API_KEY is missing or empty, the code logs a warning ("⚠️ Gemini API key not found. Falling back to Ollama.") and returns an OllamaProvider instance instead, ensuring the application continues running locally.

Can I switch between Ollama and Gemini in the same Python process?

Yes. Because ResumeEvaluator accepts a model_name parameter in its constructor, you can create multiple evaluator instances with different models. Each call to initialize_llm_provider resolves independently based on the provided model name, allowing one instance to use Ollama while another uses Gemini within the same runtime.

Where is the default provider defined if no environment variables are set?

The default provider is OLLAMA, defined as the fallback value in prompt.py via the ModelProvider enum. In llm_utils.py, the initialize_llm_provider function starts with provider = OllamaProvider() and only reassigns to GeminiProvider if the model mapping and API key checks pass.

How do I add a new Gemini model that is not in the default mapping?

Add the new model identifier to the MODEL_PROVIDER_MAPPING dictionary in prompt.py (lines 48-64), mapping it to ModelProvider.GEMINI. For example, "gemini-2.0-ultra": ModelProvider.GEMINI. Once mapped, the initialize_llm_provider function will recognize the model and attempt to instantiate GeminiProvider when that model name is requested.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}