How to Configure Gemini as an LLM Provider Instead of Ollama in the Hiring-Agent Repository

Set LLM_PROVIDER=gemini and provide your GEMINI_API_KEY in the environment; the get_llm() factory in main/llm_utils.py automatically instantiates the Gemini provider instead of Ollama.

The interviewstreet/hiring-agent repository abstracts Large Language Model interactions behind a unified LLMProvider interface. To configure Gemini as an LLM provider instead of Ollama, you modify the centralized settings in main/config.py and environment variables without altering business logic. This architecture allows seamless swapping between local Ollama instances and Google's Gemini API through a single configuration change.

Configuration Schema in main/config.py

The repository uses a ProviderConfig model defined in main/config.py to normalize settings across different LLM backends. This configuration object reads environment variables and exposes a consistent interface for the rest of the application.

Key environment variables include:

• LLM_PROVIDER – The backend selector. Use "gemini" to switch from the default Ollama setup.
• GEMINI_API_KEY – Your Google AI Studio API key (required only when provider is gemini).
• OLLAMA_BASE_URL – The local Ollama server endpoint (defaults to http://localhost:11434).
• LLM_MODEL – The model identifier, such as "gemini-pro" for Gemini or "llama2" for Ollama.

A typical .env file for Gemini configuration looks like this:

LLM_PROVIDER=gemini
GEMINI_API_KEY=your_api_key_here
LLM_MODEL=gemini-pro

main/config.py loads these values using os.getenv or Pydantic BaseSettings, making them available as config.provider, config.api_key, and config.model throughout the codebase.

Provider Implementations in main/llm_utils.py

Concrete provider implementations reside in main/llm_utils.py, where each backend encapsulates its specific authentication and request logic behind a common interface.

OllamaProvider Class

The OllamaProvider class communicates with a locally running Ollama server via HTTP:

class OllamaProvider:
    def __init__(self, base_url: str, model: str):
        self.base_url = base_url.rstrip("/")
        self.model = model

    def generate(self, prompt: str) -> str:
        resp = requests.post(
            f"{self.base_url}/api/generate",
            json={"model": self.model, "prompt": prompt},
            timeout=30,
        )
        resp.raise_for_status()
        return resp.json()["response"]

This implementation requires no API key and targets the /api/generate endpoint of the Ollama REST API.

GeminiProvider Class

The GeminiProvider class wraps the google-generativeai Python SDK:

class GeminiProvider:
    def __init__(self, api_key: str, model: str):
        self.client = genai.GenerativeModel(model)
        genai.configure(api_key=api_key)

    def generate(self, prompt: str) -> str:
        response = self.client.generate_content(prompt)
        return response.text

Both classes implement the same generate(prompt: str) -> str method signature, ensuring drop-in interchangeability.

Runtime Selection via the get_llm() Factory

The get_llm() function in main/llm_utils.py acts as a factory that instantiates the appropriate provider based on config.provider:

def get_llm():
    if config.provider == "ollama":
        return OllamaProvider(
            base_url=config.base_url or "http://localhost:11434",
            model=config.model or "llama2",
        )
    elif config.provider == "gemini":
        return GeminiProvider(
            api_key=config.api_key,
            model=config.model or "gemini-pro",
        )
    else:
        raise ValueError(f"Unsupported provider: {config.provider}")

Because the hiring pipeline calls get_llm().generate(prompt) rather than instantiating providers directly, switching from Ollama to Gemini requires only environment variable changes—no code modifications are necessary.

Step-by-Step Migration from Ollama to Gemini

Follow these steps to configure Gemini as your active LLM provider:

1. Install the Gemini SDK – Add google-generativeai to your dependencies if not already present:

   pip install google-generativeai

2. Update Environment Variables – Modify your .env file or export variables:

   export LLM_PROVIDER=gemini
   export GEMINI_API_KEY=your_actual_key_here
   export LLM_MODEL=gemini-pro

3. Verify Configuration – The application loads main/config.py at startup and validates that GEMINI_API_KEY is present when LLM_PROVIDER=gemini.

4. Run the Pipeline – Execute your hiring-agent scripts normally. The get_llm() factory automatically returns a GeminiProvider instance:

   from main.llm_utils import get_llm
   
   llm = get_llm()  # Returns GeminiProvider when configured
   
   result = llm.generate("Evaluate this candidate's Python skills.")

Programmatic Provider Switching for Testing

For unit tests or A/B comparisons, you can instantiate providers directly without using the global configuration:

from main.llm_utils import OllamaProvider, GeminiProvider

# Test against local Ollama

local_llm = OllamaProvider(
    base_url="http://localhost:11434",
    model="llama2"
)

# Test against Gemini

cloud_llm = GeminiProvider(
    api_key="your_api_key",
    model="gemini-pro"
)

# Compare outputs

local_response = local_llm.generate("Explain recursion.")
cloud_response = cloud_llm.generate("Explain recursion.")

This approach bypasses the get_llm() factory and is useful for integration testing or temporary provider overrides in Jupyter notebooks.

Summary

• Configuration is centralized in main/config.py through the ProviderConfig model, which reads LLM_PROVIDER, GEMINI_API_KEY, and LLM_MODEL from the environment.
• Provider logic is encapsulated in main/llm_utils.py, with OllamaProvider using HTTP requests to local servers and GeminiProvider using the official Google SDK.
• Runtime selection happens in the get_llm() factory function, enabling zero-code switching between backends.
• Migration requires only environment changes—set LLM_PROVIDER=gemini and provide a valid API key to move from local Ollama to Google's hosted models.

Frequently Asked Questions

Do I need to restart the application when switching providers?

Yes. The get_llm() factory typically runs once at startup when the hiring pipeline initializes. Changes to LLM_PROVIDER or GEMINI_API_KEY require a process restart to reload main/config.py and instantiate the new provider class.

Can I use both Ollama and Gemini simultaneously in different parts of the application?

Absolutely. While get_llm() returns a single global provider based on configuration, you can import OllamaProvider and GeminiProvider directly from main/llm_utils.py and instantiate them side-by-side. This is useful for fallback logic or comparative benchmarking between local and cloud models.

What happens if I configure Gemini but forget to set GEMINI_API_KEY?

The GeminiProvider constructor calls genai.configure(api_key=api_key), which will raise an authentication error or fail silently depending on the google-generativeai version. The application will crash during the first generate() call with a clear message indicating the missing API key, as enforced by the underlying SDK.

Is there a performance difference between Ollama and Gemini providers?

Yes. OllamaProvider makes synchronous HTTP requests to localhost, yielding latencies dependent on your local hardware (typically 1-10 seconds for large prompts). GeminiProvider incurs network latency to Google's API endpoints but offers higher throughput and GPU acceleration for large models. The generate() method signature remains identical, so performance optimizations can be tested by simply swapping the provider configuration.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}