# How to Configure Gemini as an LLM Provider Instead of Ollama in the Hiring-Agent Repository

> Easily configure Gemini as your LLM provider in the hiring-agent repository. Simply set LLM_PROVIDER=gemini and provide your GEMINI_API_KEY to switch from Ollama.

- Repository: [HackerRank/hiring-agent](https://github.com/interviewstreet/hiring-agent)
- Tags: how-to-guide
- Published: 2026-07-22

---

**Set `LLM_PROVIDER=gemini` and provide your `GEMINI_API_KEY` in the environment; the `get_llm()` factory in [`main/llm_utils.py`](https://github.com/interviewstreet/hiring-agent/blob/main/main/llm_utils.py) automatically instantiates the Gemini provider instead of Ollama.**

The interviewstreet/hiring-agent repository abstracts Large Language Model interactions behind a unified `LLMProvider` interface. To configure Gemini as an LLM provider instead of Ollama, you modify the centralized settings in [`main/config.py`](https://github.com/interviewstreet/hiring-agent/blob/main/main/config.py) and environment variables without altering business logic. This architecture allows seamless swapping between local Ollama instances and Google's Gemini API through a single configuration change.

## Configuration Schema in main/config.py

The repository uses a `ProviderConfig` model defined in [`main/config.py`](https://github.com/interviewstreet/hiring-agent/blob/main/main/config.py) to normalize settings across different LLM backends. This configuration object reads environment variables and exposes a consistent interface for the rest of the application.

Key environment variables include:

- **`LLM_PROVIDER`** – The backend selector. Use `"gemini"` to switch from the default Ollama setup.
- **`GEMINI_API_KEY`** – Your Google AI Studio API key (required only when provider is `gemini`).
- **`OLLAMA_BASE_URL`** – The local Ollama server endpoint (defaults to `http://localhost:11434`).
- **`LLM_MODEL`** – The model identifier, such as `"gemini-pro"` for Gemini or `"llama2"` for Ollama.

A typical `.env` file for Gemini configuration looks like this:

```bash
LLM_PROVIDER=gemini
GEMINI_API_KEY=your_api_key_here
LLM_MODEL=gemini-pro

```

[`main/config.py`](https://github.com/interviewstreet/hiring-agent/blob/main/main/config.py) loads these values using `os.getenv` or Pydantic `BaseSettings`, making them available as `config.provider`, `config.api_key`, and `config.model` throughout the codebase.

## Provider Implementations in main/llm_utils.py

Concrete provider implementations reside in [`main/llm_utils.py`](https://github.com/interviewstreet/hiring-agent/blob/main/main/llm_utils.py), where each backend encapsulates its specific authentication and request logic behind a common interface.

### OllamaProvider Class

The `OllamaProvider` class communicates with a locally running Ollama server via HTTP:

```python
class OllamaProvider:
    def __init__(self, base_url: str, model: str):
        self.base_url = base_url.rstrip("/")
        self.model = model

    def generate(self, prompt: str) -> str:
        resp = requests.post(
            f"{self.base_url}/api/generate",
            json={"model": self.model, "prompt": prompt},
            timeout=30,
        )
        resp.raise_for_status()
        return resp.json()["response"]

```

This implementation requires no API key and targets the `/api/generate` endpoint of the Ollama REST API.

### GeminiProvider Class

The `GeminiProvider` class wraps the `google-generativeai` Python SDK:

```python
class GeminiProvider:
    def __init__(self, api_key: str, model: str):
        self.client = genai.GenerativeModel(model)
        genai.configure(api_key=api_key)

    def generate(self, prompt: str) -> str:
        response = self.client.generate_content(prompt)
        return response.text

```

Both classes implement the same `generate(prompt: str) -> str` method signature, ensuring drop-in interchangeability.

## Runtime Selection via the get_llm() Factory

The `get_llm()` function in [`main/llm_utils.py`](https://github.com/interviewstreet/hiring-agent/blob/main/main/llm_utils.py) acts as a factory that instantiates the appropriate provider based on `config.provider`:

```python
def get_llm():
    if config.provider == "ollama":
        return OllamaProvider(
            base_url=config.base_url or "http://localhost:11434",
            model=config.model or "llama2",
        )
    elif config.provider == "gemini":
        return GeminiProvider(
            api_key=config.api_key,
            model=config.model or "gemini-pro",
        )
    else:
        raise ValueError(f"Unsupported provider: {config.provider}")

```

Because the hiring pipeline calls `get_llm().generate(prompt)` rather than instantiating providers directly, **switching from Ollama to Gemini requires only environment variable changes—no code modifications are necessary**.

## Step-by-Step Migration from Ollama to Gemini

Follow these steps to configure Gemini as your active LLM provider:

1. **Install the Gemini SDK** – Add `google-generativeai` to your dependencies if not already present:

   ```bash
   pip install google-generativeai
   ```

2. **Update Environment Variables** – Modify your `.env` file or export variables:

   ```bash
   export LLM_PROVIDER=gemini
   export GEMINI_API_KEY=your_actual_key_here
   export LLM_MODEL=gemini-pro
   ```

3. **Verify Configuration** – The application loads [`main/config.py`](https://github.com/interviewstreet/hiring-agent/blob/main/main/config.py) at startup and validates that `GEMINI_API_KEY` is present when `LLM_PROVIDER=gemini`.

4. **Run the Pipeline** – Execute your hiring-agent scripts normally. The `get_llm()` factory automatically returns a `GeminiProvider` instance:

   ```python
   from main.llm_utils import get_llm
   
   llm = get_llm()  # Returns GeminiProvider when configured

   result = llm.generate("Evaluate this candidate's Python skills.")
   ```

## Programmatic Provider Switching for Testing

For unit tests or A/B comparisons, you can instantiate providers directly without using the global configuration:

```python
from main.llm_utils import OllamaProvider, GeminiProvider

# Test against local Ollama

local_llm = OllamaProvider(
    base_url="http://localhost:11434",
    model="llama2"
)

# Test against Gemini

cloud_llm = GeminiProvider(
    api_key="your_api_key",
    model="gemini-pro"
)

# Compare outputs

local_response = local_llm.generate("Explain recursion.")
cloud_response = cloud_llm.generate("Explain recursion.")

```

This approach bypasses the `get_llm()` factory and is useful for integration testing or temporary provider overrides in Jupyter notebooks.

## Summary

- **Configuration is centralized** in [`main/config.py`](https://github.com/interviewstreet/hiring-agent/blob/main/main/config.py) through the `ProviderConfig` model, which reads `LLM_PROVIDER`, `GEMINI_API_KEY`, and `LLM_MODEL` from the environment.
- **Provider logic is encapsulated** in [`main/llm_utils.py`](https://github.com/interviewstreet/hiring-agent/blob/main/main/llm_utils.py), with `OllamaProvider` using HTTP requests to local servers and `GeminiProvider` using the official Google SDK.
- **Runtime selection happens** in the `get_llm()` factory function, enabling zero-code switching between backends.
- **Migration requires only environment changes**—set `LLM_PROVIDER=gemini` and provide a valid API key to move from local Ollama to Google's hosted models.

## Frequently Asked Questions

### Do I need to restart the application when switching providers?

Yes. The `get_llm()` factory typically runs once at startup when the hiring pipeline initializes. Changes to `LLM_PROVIDER` or `GEMINI_API_KEY` require a process restart to reload [`main/config.py`](https://github.com/interviewstreet/hiring-agent/blob/main/main/config.py) and instantiate the new provider class.

### Can I use both Ollama and Gemini simultaneously in different parts of the application?

Absolutely. While `get_llm()` returns a single global provider based on configuration, you can import `OllamaProvider` and `GeminiProvider` directly from [`main/llm_utils.py`](https://github.com/interviewstreet/hiring-agent/blob/main/main/llm_utils.py) and instantiate them side-by-side. This is useful for fallback logic or comparative benchmarking between local and cloud models.

### What happens if I configure Gemini but forget to set GEMINI_API_KEY?

The `GeminiProvider` constructor calls `genai.configure(api_key=api_key)`, which will raise an authentication error or fail silently depending on the `google-generativeai` version. The application will crash during the first `generate()` call with a clear message indicating the missing API key, as enforced by the underlying SDK.

### Is there a performance difference between Ollama and Gemini providers?

Yes. `OllamaProvider` makes synchronous HTTP requests to `localhost`, yielding latencies dependent on your local hardware (typically 1-10 seconds for large prompts). `GeminiProvider` incurs network latency to Google's API endpoints but offers higher throughput and GPU acceleration for large models. The `generate()` method signature remains identical, so performance optimizations can be tested by simply swapping the provider configuration.