How LLM Providers Are Abstracted in the models.py Provider Interface

The interviewstreet/hiring-agent repository abstracts LLM providers through a Protocol-based interface defined in models.py that establishes a strict contract for chat completions and embeddings, enabling seamless swapping between concrete implementations like OllamaProvider and GeminiProvider.

This abstraction layer decouples the application logic from specific LLM service implementations. By centralizing provider selection in an enum and enforcing method contracts through Python's typing.Protocol, the codebase maintains type safety while supporting multiple backend services.

The Three-Layer Abstraction Architecture

The provider interface in models.py consists of three distinct components that work together to standardize LLM interactions across the application.

ModelProvider Enum: Centralizing Service Selection

The ModelProvider enumeration (lines 6-12) defines the supported LLM backends as explicit constants. This enum serves as the single source of truth for available providers, preventing string-typos and enabling IDE autocomplete when selecting services.

class ModelProvider(Enum):
    OLLAMA = "ollama"
    GEMINI = "gemini"

According to the source code, this enum is used with a mapping dictionary (MODEL_PROVIDER_MAPPING) to translate model names into provider types, ensuring the rest of the codebase references only these standardized values.

LLMProvider Protocol: Defining the Contract

The core abstraction is the LLMProvider protocol (lines 14-30), which uses Python's structural subtyping to define the required interface without enforcing inheritance. Any class implementing these methods is considered a valid provider.

class LLMProvider(Protocol):
    def chat_completion(
        self, 
        messages: List[Dict], 
        model: str, 
        temperature: float = 0.0
    ) -> str:
        ...
    
    def embed(self, texts: List[str]) -> List[float]:
        ...

This protocol mandates two essential operations: chat_completion for generating responses and embed for creating text embeddings. The use of Protocol from the typing module allows static type checkers to validate provider implementations while maintaining runtime flexibility.

Concrete Implementations: OllamaProvider and GeminiProvider

Two concrete classes implement the LLMProvider protocol, each wrapping a specific LLM service's HTTP API.

OllamaProvider (lines 271-313) interfaces with local Ollama instances:

class OllamaProvider:
    def __init__(self, base_url: str = "http://localhost:11434") -> None:
        self.base_url = base_url
    
    def chat_completion(self, messages, model, temperature=0.0):
        # Implementation wrapping Ollama HTTP API

        ...
    
    def embed(self, texts):
        # Embedding implementation for Ollama models

        ...

GeminiProvider (lines 313-387) handles Google's Gemini REST API:

class GeminiProvider:
    def __init__(self, api_key: str):
        self.api_key = api_key
        self.base_url = "https://generativelanguage.googleapis.com"
    
    def chat_completion(self, messages, model, temperature=0.0):
        # Gemini-specific request construction and error handling

        ...
    
    def embed(self, texts):
        # Embedding implementation using Gemini API

        ...

Both classes conform to the LLMProvider protocol while handling service-specific authentication, request formatting, and error handling internally.

How the Abstraction Works in Practice

The rest of the codebase interacts with LLMs exclusively through the abstract protocol. The llm_utils.py file contains a factory function that instantiates the appropriate concrete provider based on the ModelProvider enum value:

from models import ModelProvider, OllamaProvider, GeminiProvider

def get_provider(model_name: str) -> LLMProvider:
    provider_type = MODEL_PROVIDER_MAPPING.get(model_name, ModelProvider.OLLAMA)
    
    if provider_type == ModelProvider.GEMINI:
        return GeminiProvider(api_key=os.getenv("GEMINI_API_KEY"))
    return OllamaProvider()

This factory pattern ensures that calling code remains agnostic to the specific LLM service. For example, prompt.py generates completions without knowing whether it's talking to Ollama or Gemini:

provider = get_provider("gemini-2.5-flash")
response = provider.chat_completion(
    messages=[{"role": "user", "content": "Explain recursion"}],
    model="gemini-2.5-flash",
    temperature=0.7,
)

embeddings = provider.embed(["first sentence", "second sentence"])

Integration with the Rest of the Codebase

The abstraction spans three primary files:

• models.py: Contains the ModelProvider enum, LLMProvider protocol, and concrete implementations (OllamaProvider, GeminiProvider)
• llm_utils.py: Provides the factory function that maps model names to provider instances and exposes convenience wrappers
• prompt.py: Consumes the abstract interface to generate completions for CLI prompts

This structure enables pluggability—adding a new LLM service requires only creating a new class that implements the LLMProvider protocol and registering it in the enum mapping. It also supports testability, as mock providers can implement the same protocol for unit testing without external API calls.

Summary

• The ModelProvider enum (lines 6-12) centralizes supported LLM services as type-safe constants
• The LLMProvider protocol (lines 14-30) defines the required contract for chat completions and embeddings using Python's typing.Protocol
• OllamaProvider (lines 271-313) and GeminiProvider (lines 313-387) provide concrete implementations for their respective APIs
• The abstraction enables runtime provider selection through a factory function in llm_utils.py
• Callers in prompt.py and other modules work exclusively with the protocol interface, ensuring clean separation of concerns

Frequently Asked Questions

What is the LLMProvider protocol in models.py?

The LLMProvider protocol is a Python typing.Protocol definition (lines 14-30) that specifies the interface any LLM provider must implement. It requires two methods: chat_completion for generating text responses and embed for creating vector embeddings. This protocol enables static type checking while allowing runtime polymorphism between different LLM services.

How does the ModelProvider enum determine which LLM to use?

The ModelProvider enum (lines 6-12) defines constants for each supported service (e.g., OLLAMA, GEMINI). A mapping dictionary (MODEL_PROVIDER_MAPPING) translates model name strings into these enum values. The factory function in llm_utils.py uses this enum to instantiate the correct concrete provider class, centralizing the selection logic in one place.

What methods must a concrete LLM provider implement?

Any concrete provider must implement the two methods defined in the LLMProvider protocol: chat_completion(self, messages, model, temperature=0.0) -> str and embed(self, texts) -> List[float]. Additionally, providers typically implement service-specific initialization (such as API key configuration in GeminiProvider.__init__ or base URL settings in OllamaProvider.__init__).

How do I add a new LLM provider to the hiring-agent repository?

To add a new provider, create a new class in models.py that implements the LLMProvider protocol with chat_completion and embed methods. Add a corresponding entry to the ModelProvider enum, update the MODEL_PROVIDER_MAPPING dictionary to include your model names, and implement any necessary authentication logic in the class __init__ method. The existing factory function in llm_utils.py will automatically handle instantiation once the mapping is updated.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}