# Architecture for Integrating Different LLM Providers: Ollama and Gemini Explained

> Learn the architecture for integrating LLM providers like Ollama and Gemini with interviewstreet/hiring-agent. Abstract LLM back-ends for seamless routing without vendor lock-in.

- Repository: [HackerRank/hiring-agent](https://github.com/interviewstreet/hiring-agent)
- Tags: architecture
- Published: 2026-07-19

---

**The interviewstreet/hiring-agent repository abstracts LLM back-ends behind a unified interface that exposes a single `chat` method, enabling the codebase to route requests to either Ollama or Google Gemini without vendor-specific logic.**

The architecture decouples model inference from application logic through a three-layer design consisting of a **provider enumeration**, **concrete provider classes**, and a **central factory**. This structure lives primarily in [`models.py`](https://github.com/interviewstreet/hiring-agent/blob/main/models.py), [`prompt.py`](https://github.com/interviewstreet/hiring-agent/blob/main/prompt.py), and [`llm_utils.py`](https://github.com/interviewstreet/hiring-agent/blob/main/llm_utils.py), making it straightforward to swap local Ollama instances with cloud-based Gemini APIs—or add entirely new providers—by updating configuration rather than rewriting business logic.

## Three-Layer Provider Abstraction

The integration strategy relies on three distinct layers that separate concerns between type safety, vendor-specific implementations, and runtime selection.

### Provider Enumeration

The `ModelProvider` enum in **[`models.py`](https://github.com/interviewstreet/hiring-agent/blob/main/models.py)** guarantees a finite set of supported back-ends and enables explicit mapping of model names to their respective handlers.

```python
class ModelProvider(Enum):
    OLLAMA = "ollama"
    GEMINI = "gemini"

```

This enumeration acts as the single source of truth for provider identity throughout the application. Any code checking provider type compares against these enum values rather than hard-coded strings, preventing drift between configuration and implementation.

### Concrete Provider Classes

Each LLM back-end implements a common interface defined by the `chat` method signature: `chat(model, messages, options, **kwargs)`. Both classes reside in **[`models.py`](https://github.com/interviewstreet/hiring-agent/blob/main/models.py)**.

**`OllamaProvider`** wraps the official `ollama` Python client and forwards requests with a 32,000-token context window. It accepts the generic message format and transmits it unchanged to the local Ollama server.

**`GeminiProvider`** configures `google.generativeai` with the supplied API key, translates the standard message list into Gemini’s `role`/`parts` structure, applies exponential back-off on quota errors, and converts the Gemini response back into an Ollama-style JSON payload. This normalization ensures downstream components receive a consistent data structure regardless of which vendor generated the response.

### Factory Selection Logic

The `initialize_llm_provider` function in **[`llm_utils.py`](https://github.com/interviewstreet/hiring-agent/blob/main/llm_utils.py)** serves as the central dispatcher. It reads the requested model name, consults the provider mapping, validates environment configuration (such as API keys), and returns an instantiated provider class.

```python
def initialize_llm_provider(model_name: str) -> Any:
    provider = OllamaProvider()  # default fallback

    model_provider = MODEL_PROVIDER_MAPPING.get(
        model_name, ModelProvider.OLLAMA
    )
    if model_provider == ModelProvider.GEMINI:
        if not GEMINI_API_KEY:
            logger.warning("⚠️ Gemini API key not found. Falling back to Ollama.")
        else:
            logger.info(f"🔄 Using Google Gemini API provider with model {model_name}")
            provider = GeminiProvider(api_key=GEMINI_API_KEY)
    else:
        logger.info(f"🔄 Using Ollama provider with model {model_name}")
    return provider

```

This factory guarantees that the rest of the application receives an object with a predictable `chat` method while centralizing provider-specific instantiation logic.

## Model-to-Provider Mapping Configuration

The **[`prompt.py`](https://github.com/interviewstreet/hiring-agent/blob/main/prompt.py)** file contains the `MODEL_PROVIDER_MAPPING` dictionary that explicitly associates each supported model with its provider enum value.

```python
MODEL_PROVIDER_MAPPING = {
    "qwen3:1.7b": ModelProvider.OLLAMA,
    "gemma3:4b":  ModelProvider.OLLAMA,
    "gemini-2.0-flash": ModelProvider.GEMINI,
    # … other models …

}

```

When `initialize_llm_provider` receives a model name, it looks up the corresponding `ModelProvider` value here. If the model is absent from the mapping, the system defaults to `ModelProvider.OLLAMA`, ensuring robustness against configuration drift.

## Environment-Based Configuration and Fallbacks

Configuration flows through environment variables loaded via `python-dotenv` in **[`prompt.py`](https://github.com/interviewstreet/hiring-agent/blob/main/prompt.py)**. The system respects three critical variables:

- **`DEFAULT_MODEL`** – Specifies the fallback model when none is explicitly requested.
- **`LLM_PROVIDER`** – Overrides the auto-detection mechanism in specific edge cases.
- **`GEMINI_API_KEY`** – Required for instantiating `GeminiProvider`; if missing, the factory logs a warning and returns `OllamaProvider` instead.

This graceful degradation ensures that development environments without Google Cloud credentials automatically route to local Ollama instances without crashing the application.

## Usage Flow and Polymorphic Interface

The consumer code interacts with the architecture through a four-step flow that remains identical regardless of which provider ultimately handles the request:

1. Determine the target model name (e.g., `"gemini-2.0-flash"` or `"gemma3:4b"`).
2. Call `initialize_llm_provider(model_name)` to receive a concrete provider instance.
3. Execute `provider.chat(model=model_name, messages=msg_list, options=params)`.
4. Process the standardized response dictionary containing the `"message"` field.

Because both providers return responses in the same Ollama-compatible JSON structure, downstream components—such as RAG pipelines or evaluation frameworks—remain completely agnostic to the underlying vendor.

## Extending the Architecture with New Providers

Adding support for additional LLM services requires three minimal changes:

1. **Define the provider enum value** in **[`models.py`](https://github.com/interviewstreet/hiring-agent/blob/main/models.py)** by extending `ModelProvider`.
2. **Implement the provider class** in **[`models.py`](https://github.com/interviewstreet/hiring-agent/blob/main/models.py)** with a `chat` method matching the signature `chat(self, model, messages, options=None, **kwargs)`.
3. **Update the mapping and factory** in **[`prompt.py`](https://github.com/interviewstreet/hiring-agent/blob/main/prompt.py)** and **[`llm_utils.py`](https://github.com/interviewstreet/hiring-agent/blob/main/llm_utils.py)** to recognize the new provider type and instantiate the class when appropriate.

This plug-in architecture ensures that integrations with providers like Anthropic Claude, OpenAI GPT, or Cohere Command can be added without modifying the core application logic.

## Summary

- The **three-layer architecture** (enumeration, concrete classes, factory) decouples vendor-specific SDKs from business logic in the interviewstreet/hiring-agent repository.
- **`ModelProvider`** enum and **`MODEL_PROVIDER_MAPPING`** dictionary in [`prompt.py`](https://github.com/interviewstreet/hiring-agent/blob/main/prompt.py) provide explicit, type-safe routing from model names to implementations.
- **`initialize_llm_provider`** in [`llm_utils.py`](https://github.com/interviewstreet/hiring-agent/blob/main/llm_utils.py) centralizes instantiation logic, handles environment validation, and implements graceful fallbacks to Ollama when Gemini credentials are absent.
- Both **`OllamaProvider`** and **`GeminiProvider`** expose identical `chat` method signatures and normalize responses to a common JSON format.
- The system supports **extensible configuration** through `.env` files, allowing developers to switch between local and cloud inference by changing environment variables rather than code.

## Frequently Asked Questions

### How does the factory decide between Ollama and Gemini?

The `initialize_llm_provider` function looks up the requested model name in the `MODEL_PROVIDER_MAPPING` dictionary defined in [`prompt.py`](https://github.com/interviewstreet/hiring-agent/blob/main/prompt.py). If the mapped value equals `ModelProvider.GEMINI` and the `GEMINI_API_KEY` environment variable is present, it returns a `GeminiProvider` instance; otherwise, it defaults to `OllamaProvider`.

### What happens if the Gemini API key is missing?

When the factory detects a Gemini-mapped model but finds `GEMINI_API_KEY` unset or empty, it logs a warning message ("⚠️ Gemini API key not found. Falling back to Ollama.") and returns an `OllamaProvider` instance instead. This ensures the application continues running using local inference rather than failing with an authentication error.

### Can I add custom providers to the architecture?

Yes. Create a new class in [`models.py`](https://github.com/interviewstreet/hiring-agent/blob/main/models.py) that implements the `chat(model, messages, options, **kwargs)` method, add a corresponding entry to the `ModelProvider` enum, update `MODEL_PROVIDER_MAPPING` in [`prompt.py`](https://github.com/interviewstreet/hiring-agent/blob/main/prompt.py) to map model names to your new enum value, and extend `initialize_llm_provider` in [`llm_utils.py`](https://github.com/interviewstreet/hiring-agent/blob/main/llm_utils.py) to instantiate your class when that enum value is detected.

### Where is the model-to-provider mapping defined?

The `MODEL_PROVIDER_MAPPING` dictionary resides in **[`prompt.py`](https://github.com/interviewstreet/hiring-agent/blob/main/prompt.py)**. It maps string model identifiers (such as `"gemini-2.0-flash"` or `"qwen3:1.7b"`) to their respective `ModelProvider` enum values, enabling the factory function to resolve which concrete provider class to instantiate at runtime.