# How to Set the LLM Provider in Hiring Agent: Ollama vs Gemini Configuration

> Learn how to set your LLM provider in Hiring Agent. Configure Ollama or Gemini by setting the LLM_PROVIDER env variable and GEMINI_API_KEY for Gemini.

- Repository: [HackerRank/hiring-agent](https://github.com/interviewstreet/hiring-agent)
- Tags: how-to-guide
- Published: 2026-07-11

---

**Set the `LLM_PROVIDER` environment variable to `ollama` or `gemini` in your `.env` file, and ensure `GEMINI_API_KEY` is present when using Gemini; the `initialize_llm_provider` function in [`llm_utils.py`](https://github.com/interviewstreet/hiring-agent/blob/main/llm_utils.py) handles the runtime instantiation based on the `MODEL_PROVIDER_MAPPING` table.**

The **Hiring Agent** open-source repository (`interviewstreet/hiring-agent`) supports multiple Large Language Model (LLM) backends, allowing you to choose between local inference via **Ollama** or cloud-based generation through **Google Gemini**. Understanding how to configure and switch between these providers ensures your hiring automation pipeline uses the right model for your infrastructure and privacy requirements. This guide explains the exact configuration mechanism, file locations, and initialization patterns used in the source code.

## Understanding the Provider Selection Architecture

The Hiring Agent uses a provider-agnostic architecture that decouples the LLM implementation from the business logic. The system determines which backend to use through a combination of environment variables and a static mapping table defined in the source code.

### The ModelProvider Enum (models.py)

At the core of the selection logic sits the `ModelProvider` enum defined in [[`models.py`](https://github.com/interviewstreet/hiring-agent/blob/main/models.py)](https://github.com/interviewstreet/hiring-agent/blob/main/models.py). This enum defines two possible values:

- **OLLAMA** – Represents the local Ollama server provider
- **GEMINI** – Represents the Google Gemini API provider

Additionally, [`models.py`](https://github.com/interviewstreet/hiring-agent/blob/main/models.py) declares the `LLMProvider` protocol, which both concrete providers implement. This protocol ensures that regardless of which backend is active, calling code can invoke the unified `chat` method with consistent parameters.

### Environment Variable Defaults (prompt.py)

Default values and environment variable handling reside in [[`prompt.py`](https://github.com/interviewstreet/hiring-agent/blob/main/prompt.py)](https://github.com/interviewstreet/hiring-agent/blob/main/prompt.py). This file reads the `LLM_PROVIDER` variable (defaulting to `"ollama"` if unset) and checks for the presence of `GEMINI_API_KEY`. It also contains the `MODEL_PROVIDER_MAPPING` dictionary, which maps specific model strings (e.g., `"gemma3:4b"`, `"gemini-2.5-pro"`) to their corresponding `ModelProvider` enum values.

## Configuring Ollama vs Gemini

Switching between providers requires specific environment configurations. The system validates credentials at runtime and applies fallback logic to prevent startup failures.

### Local Ollama Setup

To use **Ollama** for local inference:

1. Set `LLM_PROVIDER=ollama` in your environment
2. Ensure your Ollama server is running and accessible
3. Optionally specify a local model via `DEFAULT_MODEL` (e.g., `gemma3:4b`)

No API key is required for Ollama operation. If you omit `LLM_PROVIDER`, the system defaults to Ollama automatically.

### Google Gemini API Setup

To configure **Gemini** for cloud-based generation:

1. Set `LLM_PROVIDER=gemini` in your `.env` file or shell environment
2. Export a valid `GEMINI_API_KEY` with appropriate permissions for the Gemini API
3. Set `DEFAULT_MODEL` to a Gemini-specific identifier (e.g., `gemini-2.5-pro`)

If you specify Gemini as the provider but fail to provide `GEMINI_API_KEY`, the code logs a warning and gracefully falls back to Ollama rather than crashing.

## Runtime Provider Initialization

The [[`llm_utils.py`](https://github.com/interviewstreet/hiring-agent/blob/main/llm_utils.py)](https://github.com/interviewstreet/hiring-agent/blob/main/llm_utils.py) file contains the critical `initialize_llm_provider(model_name)` function. This function:

- Accepts a model name (e.g., `"gemini-2.5-pro"`)
- Looks up the provider type in `MODEL_PROVIDER_MAPPING`
- Instantiates either `OllamaProvider` or `GeminiProvider` based on the mapping and environment variables
- Returns an object conforming to the `LLMProvider` protocol

All higher-level application code calls this factory function rather than instantiating providers directly, ensuring consistent behavior across the codebase.

## Practical Configuration Examples

### Environment File Configuration

Create a `.env` file in your project root to configure the provider:

```text

# .env

LLM_PROVIDER=gemini          # Use "ollama" for local inference

GEMINI_API_KEY=YOUR_KEY_HERE # Required only for Gemini

DEFAULT_MODEL=gemini-2.5-pro # Optional: specify target model

```

### Programmatic Provider Initialization

You can manually initialize and use the provider in Python:

```python
from llm_utils import initialize_llm_provider

# Select a model defined in MODEL_PROVIDER_MAPPING

model = "gemini-2.5-pro"

# Returns either OllamaProvider or GeminiProvider instance

llm = initialize_llm_provider(model)

# Call the unified interface

response = llm.chat(
    model=model,
    messages=[{"role": "user", "content": "Explain the hiring process."}],
    options={}  # Model-specific parameters from MODEL_PARAMETERS

)
print(response["message"]["content"])

```

### Verifying Active Configuration

To confirm which provider is currently active:

```python
import os
from prompt import PROVIDER

print(f"The current LLM provider is: {PROVIDER}")

# Outputs: "ollama" or "gemini"

```

## Summary

- **`LLM_PROVIDER`** environment variable controls which backend the Hiring Agent uses at runtime.
- **[`prompt.py`](https://github.com/interviewstreet/hiring-agent/blob/main/prompt.py)** defines the default values and `MODEL_PROVIDER_MAPPING` that associates model names with provider types.
- **`initialize_llm_provider`** in [`llm_utils.py`](https://github.com/interviewstreet/hiring-agent/blob/main/llm_utils.py) is the factory function that creates the appropriate provider instance based on your configuration.
- **Gemini requires** `GEMINI_API_KEY`; if missing, the system falls back to Ollama.
- **Ollama requires** no API key and serves as the default when no configuration is provided.

## Frequently Asked Questions

### How do I switch from Ollama to Gemini without modifying code?

Set `LLM_PROVIDER=gemini` in your environment or `.env` file and ensure `GEMINI_API_KEY` contains a valid Google API key. The `initialize_llm_provider` function automatically detects this change on the next application start and instantiates the `GeminiProvider` instead of `OllamaProvider`.

### What happens if I forget to set GEMINI_API_KEY when using Gemini?

The code in [`prompt.py`](https://github.com/interviewstreet/hiring-agent/blob/main/prompt.py) detects the missing credential, logs a warning message, and falls back to the Ollama provider. This ensures the Hiring Agent remains functional even when API keys are temporarily unavailable, defaulting to local inference.

### Where is the list of supported models for each provider?

The supported models are defined in the `MODEL_PROVIDER_MAPPING` dictionary located in [[`prompt.py`](https://github.com/interviewstreet/hiring-agent/blob/main/prompt.py)](https://github.com/interviewstreet/hiring-agent/blob/main/prompt.py). This mapping connects specific model strings like `"gemma3:4b"` or `"gemini-2.5-pro"` to the `OLLAMA` or `GEMINI` enum values from [[`models.py`](https://github.com/interviewstreet/hiring-agent/blob/main/models.py)](https://github.com/interviewstreet/hiring-agent/blob/main/models.py).

### Can I use both providers simultaneously in the same application instance?

While the environment variable determines the default provider, you can instantiate specific providers manually by importing the classes directly from [`llm_utils.py`](https://github.com/interviewstreet/hiring-agent/blob/main/llm_utils.py). However, the standard `initialize_llm_provider` factory function returns a single configured instance based on the global `LLM_PROVIDER` setting.