How to Configure and Switch Between Ollama and Gemini LLM Providers in Hiring-Agent

The Hiring-Agent repository abstracts LLM providers behind a unified interface in main/llm_utils.py, allowing you to configure and switch between Ollama and Gemini by updating environment variables in main/config.py without modifying application code.

The interviewstreet/hiring-agent project provides a flexible abstraction layer for Large Language Model integrations. By centralizing provider configuration in main/config.py and implementing a factory pattern in main/llm_utils.py, you can seamlessly configure and switch between Ollama and Gemini LLM providers using only environment variables.

Understanding the LLM Provider Architecture

ProviderConfig in main/config.py

The configuration system centers on the ProviderConfig class defined in main/config.py. This Pydantic model encapsulates all provider-specific settings, exposing four critical fields: provider (the service name), api_key (authentication credentials), base_url (server endpoint), and model (the specific model identifier).

The Unified LLMProvider Interface

Both Ollama and Gemini implementations conform to a consistent interface exposing two primary methods: generate(prompt: str) -> str for synchronous text generation and stream(prompt: str) -> Iterator[str] for streaming responses. This standardization ensures that switching providers requires no changes to calling code in main/prompt.py or other downstream modules.

Configuring Ollama and Gemini Providers

Environment Variables Setup

The main/config.py file loads configuration from environment variables using os.getenv or Pydantic's BaseSettings. Configure your environment using these variables:

• LLM_PROVIDER – Set to "ollama" or "gemini"
• OLLAMA_BASE_URL – Base URL for the local Ollama server (defaults to http://localhost:11434)
• GEMINI_API_KEY – API key for Google Gemini authentication
• LLM_MODEL – Model identifier such as "llama2" or "gemini-pro"

Provider-Specific Requirements

Ollama requires only the base_url and model fields since it runs locally without authentication. Gemini requires the api_key field populated with a valid Google API key, while the base_url is managed internally by the google-generativeai library.

Switching Between LLM Providers

Runtime Selection via get_llm()

The get_llm() function in main/llm_utils.py serves as the central factory for provider instantiation. This function reads config.provider from the singleton ProviderConfig instance and returns the appropriate implementation:

def get_llm() -> LLMProvider:
    if config.provider == "ollama":
        return OllamaProvider(
            base_url=config.base_url or "http://localhost:11434",
            model=config.model or "llama2",
        )
    elif config.provider == "gemini":
        return GeminiProvider(
            api_key=config.api_key,
            model=config.model or "gemini-pro",
        )
    else:
        raise ValueError(f"Unsupported provider: {config.provider}")

Ollama Implementation Details

The OllamaProvider class communicates with the local Ollama server via HTTP POST requests to the /api/generate endpoint. The implementation constructs a JSON payload containing the model name and prompt, sending it to the configured base_url.

Gemini Implementation Details

The GeminiProvider class utilizes the google-generativeai Python package. It configures the API key via genai.configure() and invokes generate_content() on the specified model instance. This implementation requires valid GEMINI_API_KEY credentials and handles cloud-based authentication automatically.

Practical Configuration Examples

Example 1: Ollama Configuration

Configure your .env file for local Ollama usage:

LLM_PROVIDER=ollama
OLLAMA_BASE_URL=http://localhost:11434
LLM_MODEL=llama2

Access the provider in your application:

from main.config import config
from main.llm_utils import get_llm

print(f"Active provider: {config.provider}")
response = get_llm().generate("Explain Python decorators.")
print(response)

Example 2: Gemini Configuration

Switch to Gemini by updating the environment:

LLM_PROVIDER=gemini
GEMINI_API_KEY=your_api_key_here
LLM_MODEL=gemini-pro

Example 3: Programmatic Provider Switching

For testing or multi-provider workflows, instantiate classes directly:

from main.llm_utils import OllamaProvider, GeminiProvider

# Initialize Ollama

ollama = OllamaProvider(base_url="http://localhost:11434", model="llama2")
local_result = ollama.generate("What is recursion?")

# Switch to Gemini

gemini = GeminiProvider(api_key="YOUR_GEMINI_KEY", model="gemini-pro")
cloud_result = gemini.generate("What is recursion?")

Summary

• The ProviderConfig class in main/config.py centralizes all LLM provider settings through environment variables.
• The get_llm() factory function in main/llm_utils.py handles runtime provider selection based on the LLM_PROVIDER environment variable.
• Ollama requires OLLAMA_BASE_URL but no authentication, while Gemini requires GEMINI_API_KEY for cloud access.
• Both providers implement the same interface, ensuring compatibility across the hiring-agent codebase without code changes.
• The factory raises a ValueError for unsupported provider names, preventing runtime misconfiguration.

Frequently Asked Questions

How do I switch from Ollama to Gemini without modifying code?

Update the LLM_PROVIDER environment variable from "ollama" to "gemini" in your .env file or shell environment, ensure GEMINI_API_KEY is set, and restart the application. The get_llm() factory in main/llm_utils.py automatically instantiates the correct provider class based on the configuration loaded from main/config.py.

What are the required environment variables for each provider?

For Ollama, you need LLM_PROVIDER=ollama, OLLAMA_BASE_URL (defaults to http://localhost:11434), and LLM_MODEL. For Gemini, you need LLM_PROVIDER=gemini, GEMINI_API_KEY, and LLM_MODEL. The api_key field is ignored for Ollama since it runs locally without authentication.

Can I use multiple LLM providers simultaneously in the same script?

Yes. While get_llm() returns a single configured provider based on environment variables, you can instantiate both classes directly by importing OllamaProvider and GeminiProvider from main/llm_utils.py. This allows you to compare outputs or route different prompts to different providers within the same execution context.

Where is the provider configuration validated?

Validation occurs in main/config.py where the ProviderConfig class validates field types, and additionally in main/llm_utils.py where the get_llm() function raises a ValueError if the provider name does not match "ollama" or "gemini". This ensures misconfigurations are caught immediately at startup rather than during prompt execution.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}