What is the LLM_PROVIDER Environment Variable in Hiring Agent?

The LLM_PROVIDER environment variable controls which large language model backend the Hiring Agent uses, defaulting to ollama but supporting gemini as an alternative.

The LLM_PROVIDER environment variable is the primary configuration switch in the interviewstreet/hiring-agent repository that determines whether the application uses a local Ollama instance or Google's Gemini API for text generation tasks. This variable is read at startup in prompt.py and drives the selection of client implementations, authentication methods, and prompt formatting throughout the codebase.

How LLM_PROVIDER Works

Configuration Resolution in prompt.py

The variable is evaluated at import time in prompt.py using the following logic:

PROVIDER = os.getenv("LLM_PROVIDER", DEFAULT_PROVIDER.value)

Where DEFAULT_PROVIDER defaults to the string "ollama". This means if you do not set the variable explicitly, the system automatically selects the local Ollama backend.

Supported Provider Values

The Hiring Agent currently supports two distinct values for LLM_PROVIDER:

• ollama: Routes requests to a locally running Ollama server using open-source models (default behavior)
• gemini: Routes requests to Google's Gemini API, requiring additional authentication configuration

When LLM_PROVIDER=gemini, the system expects a valid GEMINI_API_KEY environment variable to be present according to the repository's README.md documentation.

Configuration Methods

Local Environment File (.env)

Create or modify your .env file in the project root to persist the configuration:

LLM_PROVIDER=gemini
GEMINI_API_KEY=your_api_key_here

For local development with Ollama, you only need:

LLM_PROVIDER=ollama

Runtime Command Line Override

You can override the provider for a single execution without modifying configuration files:

LLM_PROVIDER=ollama python run_hiring_agent.py

This approach is useful for testing different backends or running specific scripts against alternate models.

Implementation Details

Client Selection in llm_utils.py

The value of LLM_PROVIDER determines which client implementation is instantiated in llm_utils.py. The code logic distinguishes between providers to initialize the appropriate connection handler:

if provider == "gemini":
    # Gemini-specific initialization

    client = GeminiClient(api_key=os.getenv("GEMINI_API_KEY"))
else:
    # Default Ollama initialization

    client = OllamaClient()

Behavioral Impact on Text Generation

The selected provider affects multiple aspects of the LLM integration:

• Endpoint configuration: Ollama uses local server URLs (typically http://localhost:11434) while Gemini uses Google Cloud API endpoints
• Authentication: Ollama requires no authentication (local deployment), whereas Gemini validates the GEMINI_API_KEY header
• Prompt formatting: Different system-message conventions and tokenization rules between the Ollama and Gemini implementations
• Model parameters: Provider-specific hyperparameters exposed through the respective client classes

Code Examples

Accessing the Provider Runtime Configuration

import os
from llm_utils import get_llm_client

# Reads LLM_PROVIDER with fallback to "ollama"

provider = os.getenv("LLM_PROVIDER", "ollama")

# Returns the appropriate client instance

client = get_llm_client(provider)
response = client.complete(prompt="Analyze this resume for Python experience.")
print(response)

Conditional Provider Logic

When building extensions that need provider-specific behavior:

provider = os.getenv("LLM_PROVIDER", "ollama")

if provider == "gemini":
    # Configure Gemini-specific settings

    model_name = "gemini-pro"
    temperature = 0.7
else:
    # Ollama defaults

    model_name = "llama2"
    temperature = 0.8

Summary

• Primary purpose: The LLM_PROVIDER environment variable switches between Ollama (local) and Gemini (cloud) LLM backends
• Default behavior: If unset, the system defaults to ollama as defined in prompt.py
• Authentication requirement: GEMINI_API_KEY is mandatory only when LLM_PROVIDER=gemini
• Key files: Configuration is read in prompt.py, client instantiation occurs in llm_utils.py, and documentation resides in README.md and .env.example

Frequently Asked Questions

What is the default value of LLM_PROVIDER?

If you do not set the LLM_PROVIDER environment variable, the Hiring Agent defaults to "ollama" according to the DEFAULT_PROVIDER constant defined in prompt.py. This ensures the application works out-of-the-box with local Ollama installations without requiring any API keys.

Do I need an API key when using LLM_PROVIDER?

You only need an API key when LLM_PROVIDER=gemini. The Ollama provider uses local inference and requires no authentication. When using Gemini, you must export GEMINI_API_KEY with a valid Google AI API key, as documented in the repository's README.md and .env.example files.

Can I use OpenAI or other providers with LLM_PROVIDER?

Currently, the Hiring Agent only supports ollama and gemini as valid values for LLM_PROVIDER. The client selection logic in llm_utils.py implements specific handling for these two providers only. To use OpenAI or other services, you would need to modify the source code to add additional provider branches.

Where is the LLM_PROVIDER variable actually read?

The variable is read at module import time in prompt.py using os.getenv("LLM_PROVIDER", DEFAULT_PROVIDER.value). This occurs when the Python process starts, meaning changes to the environment variable require a restart of the application to take effect.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}