How the Hiring Agent Supports Switching Between Different LLM Providers

The Hiring Agent supports switching between different LLM providers through a configuration-driven factory pattern that maps model names to specific provider implementations, defaulting to local Ollama inference while conditionally routing to Google Gemini when API credentials are present.

The interviewstreet/hiring-agent repository abstracts the underlying large-language-model service behind a uniform interface. This architecture allows core business logic—such as resume parsing and candidate scoring—to remain unchanged while the actual model provider is swapped at runtime based on configuration.

Configuration-Driven Provider Mapping

The switching mechanism begins with a static mapping defined in prompt.py. The MODEL_PROVIDER_MAPPING dictionary associates user-requested model identifiers with specific ModelProvider enum values.

# Defined in prompt.py

MODEL_PROVIDER_MAPPING = {
    "gemini-1.5-pro": ModelProvider.GEMINI,
    "gemini-1.5-flash": ModelProvider.GEMINI,
    # Local Ollama models typically omit explicit mapping

}

class ModelProvider(Enum):
    OLLAMA = "ollama"
    GEMINI = "gemini"

When a model name is not present in this mapping, the system defaults to ModelProvider.OLLAMA, ensuring local inference works out-of-the-box without explicit configuration.

The Provider Factory Implementation

The initialize_llm_provider function in llm_utils.py serves as the central factory that instantiates the correct provider class at runtime.

def initialize_llm_provider(model_name: str) -> Any:
    """
    Choose the proper LLM provider (Ollama or Gemini) based on the
    supplied model name.
    """
    provider = OllamaProvider()
    model_provider = MODEL_PROVIDER_MAPPING.get(model_name, ModelProvider.OLLAMA)

    if model_provider == ModelProvider.GEMINI:
        if not GEMINI_API_KEY:
            logger.warning("⚠️ Gemini API key not found. Falling back to Ollama.")
        else:
            logger.info(f"🔄 Using Google Gemini API provider with model {model_name}")
            provider = GeminiProvider(api_key=GEMINI_API_KEY)
    else:
        logger.info(f"🔄 Using Ollama provider with model {model_name}")

    return provider

Key aspects of this implementation:

• Fail-Safe Defaulting: The function initializes with OllamaProvider() as the default before checking configuration, ensuring the application remains functional even if Gemini credentials are missing.
• Environment Validation: When the mapping resolves to ModelProvider.GEMINI, the function validates the presence of the GEMINI_API_KEY environment variable before instantiation.
• Graceful Degradation: If the Gemini API key is absent, the function logs a warning and returns the Ollama provider, preventing runtime crashes while alerting operators to the configuration issue.

Unified Interface Architecture

Both providers implement the LLMProvider protocol defined in models.py, exposing a consistent chat method signature that accepts model, messages, and optional options parameters.

# models.py

class LLMProvider(Protocol):
    def chat(self, model: str, messages: List[Dict], options: Optional[Dict] = None) -> Dict:
        ...

Concrete implementations:

• OllamaProvider: Communicates with a local Ollama daemon via HTTP, ideal for air-gapped environments or cost-sensitive deployments.
• GeminiProvider: Wraps the Google Gemini REST API, handling authentication headers and request formatting transparently.

This protocol-based design means consuming code calls provider.chat() without import-time dependencies on specific vendor SDKs, achieving true provider-agnostic operation.

Runtime Provider Selection Example

The following pattern demonstrates how the same codebase switches between local and cloud inference without modification to business logic:

from llm_utils import initialize_llm_provider
from prompt import DEFAULT_MODEL  # e.g., "gemini-1.5-pro" or "llama3.1-8b"

# 1. Factory selects provider based on model name

provider = initialize_llm_provider(DEFAULT_MODEL)

# 2. Prepare standardized message payload

messages = [
    {"role": "system", "content": "You are a helpful hiring assistant."},
    {"role": "user", "content": "Summarize this resume."}
]

# 3. Unified interface executes regardless of underlying provider

response = provider.chat(
    model=DEFAULT_MODEL,
    messages=messages,
    options={"temperature": 0.7}
)

content = response["message"]["content"]

If DEFAULT_MODEL maps to a Gemini identifier and GEMINI_API_KEY is set, the request routes to Google's API; otherwise, it automatically falls back to the local Ollama instance.

Summary

• Configuration Mapping: The MODEL_PROVIDER_MAPPING dictionary in prompt.py binds model names to ModelProvider enum values, driving the selection logic.
• Factory Pattern: initialize_llm_provider() in llm_utils.py encapsulates instantiation logic, defaulting to OllamaProvider while conditionally creating GeminiProvider when credentials exist.
• Protocol Abstraction: Both providers implement the LLMProvider protocol from models.py, exposing identical chat() signatures that abstract vendor-specific implementations.
• Environment-Based Switching: The system reads GEMINI_API_KEY at runtime to determine cloud eligibility, requiring zero code changes to switch between local and hosted inference.

Frequently Asked Questions

How do I add support for a new LLM provider like OpenAI?

Extend the ModelProvider enum in models.py with a new member (e.g., OPENAI = "openai"), create a concrete class implementing the LLMProvider protocol with a chat() method, and update MODEL_PROVIDER_MAPPING in prompt.py to map specific model names to your new enum value. Finally, modify initialize_llm_provider in llm_utils.py to instantiate your new class when that enum is detected.

Why does the system default to Ollama instead of failing when Gemini is unavailable?

The architecture prioritizes fail-safe operation for hiring workflows that may run in secure environments without internet access. By defaulting to Ollama, the system ensures the application remains functional for resume processing and candidate evaluation even when external API credentials are not configured, while logging warnings to indicate the fallback state.

Can I force the system to use a specific provider regardless of the model name?

Yes, you can bypass the mapping logic by instantiating the provider class directly. Instead of calling initialize_llm_provider(), import OllamaProvider or GeminiProvider from models.py and instantiate them directly with the required parameters. However, this sacrifices the runtime flexibility that the factory pattern provides.

Where should I store the Gemini API key for the Hiring Agent to detect it?

The system expects the Gemini API key in the GEMINI_API_KEY environment variable, which is imported in prompt.py and checked within initialize_llm_provider(). Set this variable in your shell environment or .env file before starting the application; if the variable is undefined or empty, the system automatically falls back to Ollama.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}