# How LLM Providers Are Abstracted in the models.py Provider Interface

> Discover how the hiring agent abstracts LLM providers via the models.py interface. Learn about seamless provider swapping for chat completions and embeddings.

- Repository: [HackerRank/hiring-agent](https://github.com/interviewstreet/hiring-agent)
- Tags: internals
- Published: 2026-07-14

---

**The interviewstreet/hiring-agent repository abstracts LLM providers through a Protocol-based interface defined in [`models.py`](https://github.com/interviewstreet/hiring-agent/blob/main/models.py) that establishes a strict contract for chat completions and embeddings, enabling seamless swapping between concrete implementations like `OllamaProvider` and `GeminiProvider`.**

This abstraction layer decouples the application logic from specific LLM service implementations. By centralizing provider selection in an enum and enforcing method contracts through Python's `typing.Protocol`, the codebase maintains type safety while supporting multiple backend services.

## The Three-Layer Abstraction Architecture

The provider interface in [`models.py`](https://github.com/interviewstreet/hiring-agent/blob/main/models.py) consists of three distinct components that work together to standardize LLM interactions across the application.

### ModelProvider Enum: Centralizing Service Selection

The `ModelProvider` enumeration (lines 6-12) defines the supported LLM backends as explicit constants. This enum serves as the single source of truth for available providers, preventing string-typos and enabling IDE autocomplete when selecting services.

```python
class ModelProvider(Enum):
    OLLAMA = "ollama"
    GEMINI = "gemini"

```

According to the source code, this enum is used with a mapping dictionary (`MODEL_PROVIDER_MAPPING`) to translate model names into provider types, ensuring the rest of the codebase references only these standardized values.

### LLMProvider Protocol: Defining the Contract

The core abstraction is the `LLMProvider` protocol (lines 14-30), which uses Python's structural subtyping to define the required interface without enforcing inheritance. Any class implementing these methods is considered a valid provider.

```python
class LLMProvider(Protocol):
    def chat_completion(
        self, 
        messages: List[Dict], 
        model: str, 
        temperature: float = 0.0
    ) -> str:
        ...
    
    def embed(self, texts: List[str]) -> List[float]:
        ...

```

This protocol mandates two essential operations: `chat_completion` for generating responses and `embed` for creating text embeddings. The use of `Protocol` from the `typing` module allows static type checkers to validate provider implementations while maintaining runtime flexibility.

### Concrete Implementations: OllamaProvider and GeminiProvider

Two concrete classes implement the `LLMProvider` protocol, each wrapping a specific LLM service's HTTP API.

**OllamaProvider** (lines 271-313) interfaces with local Ollama instances:

```python
class OllamaProvider:
    def __init__(self, base_url: str = "http://localhost:11434") -> None:
        self.base_url = base_url
    
    def chat_completion(self, messages, model, temperature=0.0):
        # Implementation wrapping Ollama HTTP API

        ...
    
    def embed(self, texts):
        # Embedding implementation for Ollama models

        ...

```

**GeminiProvider** (lines 313-387) handles Google's Gemini REST API:

```python
class GeminiProvider:
    def __init__(self, api_key: str):
        self.api_key = api_key
        self.base_url = "https://generativelanguage.googleapis.com"
    
    def chat_completion(self, messages, model, temperature=0.0):
        # Gemini-specific request construction and error handling

        ...
    
    def embed(self, texts):
        # Embedding implementation using Gemini API

        ...

```

Both classes conform to the `LLMProvider` protocol while handling service-specific authentication, request formatting, and error handling internally.

## How the Abstraction Works in Practice

The rest of the codebase interacts with LLMs exclusively through the abstract protocol. The [`llm_utils.py`](https://github.com/interviewstreet/hiring-agent/blob/main/llm_utils.py) file contains a factory function that instantiates the appropriate concrete provider based on the `ModelProvider` enum value:

```python
from models import ModelProvider, OllamaProvider, GeminiProvider

def get_provider(model_name: str) -> LLMProvider:
    provider_type = MODEL_PROVIDER_MAPPING.get(model_name, ModelProvider.OLLAMA)
    
    if provider_type == ModelProvider.GEMINI:
        return GeminiProvider(api_key=os.getenv("GEMINI_API_KEY"))
    return OllamaProvider()

```

This factory pattern ensures that calling code remains agnostic to the specific LLM service. For example, [`prompt.py`](https://github.com/interviewstreet/hiring-agent/blob/main/prompt.py) generates completions without knowing whether it's talking to Ollama or Gemini:

```python
provider = get_provider("gemini-2.5-flash")
response = provider.chat_completion(
    messages=[{"role": "user", "content": "Explain recursion"}],
    model="gemini-2.5-flash",
    temperature=0.7,
)

embeddings = provider.embed(["first sentence", "second sentence"])

```

## Integration with the Rest of the Codebase

The abstraction spans three primary files:

- **[`models.py`](https://github.com/interviewstreet/hiring-agent/blob/main/models.py)**: Contains the `ModelProvider` enum, `LLMProvider` protocol, and concrete implementations (`OllamaProvider`, `GeminiProvider`)
- **[`llm_utils.py`](https://github.com/interviewstreet/hiring-agent/blob/main/llm_utils.py)**: Provides the factory function that maps model names to provider instances and exposes convenience wrappers
- **[`prompt.py`](https://github.com/interviewstreet/hiring-agent/blob/main/prompt.py)**: Consumes the abstract interface to generate completions for CLI prompts

This structure enables **pluggability**—adding a new LLM service requires only creating a new class that implements the `LLMProvider` protocol and registering it in the enum mapping. It also supports **testability**, as mock providers can implement the same protocol for unit testing without external API calls.

## Summary

- The `ModelProvider` enum (lines 6-12) centralizes supported LLM services as type-safe constants
- The `LLMProvider` protocol (lines 14-30) defines the required contract for chat completions and embeddings using Python's `typing.Protocol`
- `OllamaProvider` (lines 271-313) and `GeminiProvider` (lines 313-387) provide concrete implementations for their respective APIs
- The abstraction enables runtime provider selection through a factory function in [`llm_utils.py`](https://github.com/interviewstreet/hiring-agent/blob/main/llm_utils.py)
- Callers in [`prompt.py`](https://github.com/interviewstreet/hiring-agent/blob/main/prompt.py) and other modules work exclusively with the protocol interface, ensuring clean separation of concerns

## Frequently Asked Questions

### What is the LLMProvider protocol in models.py?

The `LLMProvider` protocol is a Python `typing.Protocol` definition (lines 14-30) that specifies the interface any LLM provider must implement. It requires two methods: `chat_completion` for generating text responses and `embed` for creating vector embeddings. This protocol enables static type checking while allowing runtime polymorphism between different LLM services.

### How does the ModelProvider enum determine which LLM to use?

The `ModelProvider` enum (lines 6-12) defines constants for each supported service (e.g., `OLLAMA`, `GEMINI`). A mapping dictionary (`MODEL_PROVIDER_MAPPING`) translates model name strings into these enum values. The factory function in [`llm_utils.py`](https://github.com/interviewstreet/hiring-agent/blob/main/llm_utils.py) uses this enum to instantiate the correct concrete provider class, centralizing the selection logic in one place.

### What methods must a concrete LLM provider implement?

Any concrete provider must implement the two methods defined in the `LLMProvider` protocol: `chat_completion(self, messages, model, temperature=0.0) -> str` and `embed(self, texts) -> List[float]`. Additionally, providers typically implement service-specific initialization (such as API key configuration in `GeminiProvider.__init__` or base URL settings in `OllamaProvider.__init__`).

### How do I add a new LLM provider to the hiring-agent repository?

To add a new provider, create a new class in [`models.py`](https://github.com/interviewstreet/hiring-agent/blob/main/models.py) that implements the `LLMProvider` protocol with `chat_completion` and `embed` methods. Add a corresponding entry to the `ModelProvider` enum, update the `MODEL_PROVIDER_MAPPING` dictionary to include your model names, and implement any necessary authentication logic in the class `__init__` method. The existing factory function in [`llm_utils.py`](https://github.com/interviewstreet/hiring-agent/blob/main/llm_utils.py) will automatically handle instantiation once the mapping is updated.