# Supported LLM Providers in LogSentinelAI and How to Configure Them

> Discover supported LLM providers in LogSentinelAI, including Ollama, vLLM, OpenAI, and Gemini. Learn how to easily configure them using environment variables for seamless integration.

- Repository: [JungJungIn/logsentinelai](https://github.com/call518/logsentinelai)
- Tags: how-to-guide
- Published: 2026-02-26

---

**LogSentinelAI supports four LLM backends—Ollama, vLLM, OpenAI, and Gemini—each configured via environment variables loaded by [`src/logsentinelai/core/config.py`](https://github.com/call518/logsentinelai/blob/main/src/logsentinelai/core/config.py) and instantiated through the factory function `initialize_llm_model` in [`src/logsentinelai/core/llm.py`](https://github.com/call518/logsentinelai/blob/main/src/logsentinelai/core/llm.py).**

LogSentinelAI is an open-source log analysis framework that abstracts LLM interactions through a unified client interface. Understanding the supported LLM providers in LogSentinelAI and how to configure them allows you to route log analysis tasks to local models via Ollama or vLLM, or to cloud endpoints like OpenAI and Gemini without modifying application code.

## Supported LLM Providers Overview

The `initialize_llm_model` function in [`src/logsentinelai/core/llm.py`](https://github.com/call518/logsentinelai/blob/main/src/logsentinelai/core/llm.py) (lines 20-70) routes requests to four distinct backends. All providers use the **OpenAI SDK** client with customized `base_url` and authentication parameters.

### Ollama (Local Execution)

**Ollama** enables on-premise model serving. The client wraps `openai.OpenAI` with a dummy API key and points to your local Ollama server.

- **Default Model**: `qwen2.5-coder:3b`
- **Default Host**: `http://127.0.0.1:11434/v1`
- **Environment Variables**: `LLM_API_HOST_OLLAMA`, `LLM_MODEL_OLLAMA`

### vLLM (High-Throughput Serving)

**vLLM** supports optimized local or remote model hosting. Like Ollama, it uses the OpenAI client shim but targets vLLM inference endpoints.

- **Default Model**: `Qwen/Qwen2.5-1.5B-Instruct`
- **Default Host**: `http://127.0.0.1:5000/v1`
- **Environment Variables**: `LLM_API_HOST_VLLM`, `LLM_MODEL_VLLM`

### OpenAI (Cloud API)

**OpenAI** connects to the official API using your account credentials. The SDK reads the secret directly from the environment.

- **Default Model**: `gpt-4o-mini`
- **Default Host**: `https://api.openai.com/v1`
- **Required Secret**: `OPENAI_API_KEY`

### Gemini (Google Cloud)

**Gemini** routes through Google's Generative Language API using an OpenAI-compatible shim.

- **Default Model**: `gemini-1.5-pro`
- **Default Host**: `https://generativelanguage.googleapis.com/v1beta/openai/`
- **Required Secret**: `GEMINI_API_KEY`

## Configuration Environment Variables

All provider settings are loaded at runtime from environment variables by the `_load_values` helper in [`src/logsentinelai/core/config.py`](https://github.com/call518/logsentinelai/blob/main/src/logsentinelai/core/config.py) (lines 61-73). The global provider selector defaults to `openai` if unspecified.

```python
LLM_PROVIDER = os.getenv("LLM_PROVIDER", "openai")
LLM_MODELS = {
    "ollama": os.getenv("LLM_MODEL_OLLAMA", "qwen2.5-coder:3b"),
    "vllm":   os.getenv("LLM_MODEL_VLLM",   "Qwen/Qwen2.5-1.5B-Instruct"),
    "openai": os.getenv("LLM_MODEL_OPENAI", "gpt-4o-mini"),
    "gemini": os.getenv("LLM_MODEL_GEMINI", "gemini-1.5-pro"),
}
LLM_API_HOSTS = {
    "ollama": os.getenv("LLM_API_HOST_OLLAMA", "http://127.0.0.1:11434/v1"),
    "vllm":   os.getenv("LLM_API_HOST_VLLM",   "http://127.0.0.1:5000/v1"),
    "openai": os.getenv("LLM_API_HOST_OPENAI", "https://api.openai.com/v1"),
    "gemini": os.getenv("LLM_API_HOST_GEMINI", "https://generativelanguage.googleapis.com/v1beta/openai/"),
}

```

## Step-by-Step Configuration Guide

1. **Copy the environment template** from the repository root:

   ```bash
   cp .env.template .env
   ```

2. **Set the global provider** (optional—defaults to `openai`):

   ```dotenv
   LLM_PROVIDER=ollama          # or vllm, openai, gemini

   ```

3. **Override model names** (optional):

   ```dotenv
   LLM_MODEL_OLLAMA=qwen2.5-coder:3b
   LLM_MODEL_VLLM=Qwen/Qwen2.5-1.5B-Instruct
   LLM_MODEL_OPENAI=gpt-4o-mini
   LLM_MODEL_GEMINI=gemini-1.5-pro
   ```

4. **Configure endpoint URLs** for self-hosted instances:

   ```dotenv
   LLM_API_HOST_OLLAMA=http://127.0.0.1:11434/v1
   LLM_API_HOST_VLLM=http://127.0.0.1:5000/v1
   ```

5. **Provide authentication tokens** for cloud providers:

   ```dotenv
   OPENAI_API_KEY=sk-...
   GEMINI_API_KEY=AIza...
   ```

6. **Reload configuration** at runtime after changes:

   ```python
   from logsentinelai.core.config import apply_config
   apply_config()
   ```

## Initializing and Using LLM Clients in Code

The factory function `initialize_llm_model` instantiates the correct client based on the provider string. Use `generate_with_model` for unified inference across all backends, including special post-processing for Gemini responses.

```python
from logsentinelai.core.llm import initialize_llm_model, generate_with_model
from logsentinelai.core.config import LLM_PROVIDER, LLM_MODELS
from pydantic import BaseModel

# 1. Select provider (overrides .env if needed)

provider = "gemini"                       # could be "ollama", "vllm", "openai", "gemini"

model_name = LLM_MODELS[provider]

# 2. Build the model object

model = initialize_llm_model(llm_provider=provider, llm_model_name=model_name)

# 3. Define a Pydantic schema for structured output

class IssueReport(BaseModel):
    level: str
    description: str
    timestamp: str

# 4. Send a prompt and get validated JSON

prompt = "Summarize the most critical error in the following log snippet..."
json_result = generate_with_model(model, prompt, IssueReport, llm_provider=provider)

print(json_result)   # → validated JSON string

```

This implementation works for any of the four providers because `generate_with_model` adapts the call internally—Gemini receives markdown stripping and validation, while Ollama, vLLM, and OpenAI use the standard `outlines` wrapper.

## Provider-Specific Implementation Details

### Client Instantiation Logic

In [`src/logsentinelai/core/llm.py`](https://github.com/call518/logsentinelai/blob/main/src/logsentinelai/core/llm.py), the `initialize_llm_model` function (lines 20-70) matches the provider string and constructs the appropriate `openai.OpenAI` instance. **Ollama** and **vLLM** use dummy API keys with custom `base_url` parameters, while **OpenAI** and **Gemini** pass real authentication headers via environment variables.

### Response Handling Differences

While Ollama, vLLM, and OpenAI use standard JSON schema enforcement, **Gemini** requires special post-processing. The `generate_with_model` function strips markdown formatting from Gemini responses before Pydantic validation, ensuring consistent output structure across all providers.

## Summary

- LogSentinelAI supports **four LLM providers**: Ollama, vLLM, OpenAI, and Gemini
- Configuration occurs through **environment variables** defined in [`src/logsentinelai/core/config.py`](https://github.com/call518/logsentinelai/blob/main/src/logsentinelai/core/config.py)
- Local providers require **host URL configuration**, while cloud providers need **API keys** (`OPENAI_API_KEY` or `GEMINI_API_KEY`)
- The `initialize_llm_model` factory in [`src/logsentinelai/core/llm.py`](https://github.com/call518/logsentinelai/blob/main/src/logsentinelai/core/llm.py) abstracts client creation and routing
- Runtime configuration changes require calling **`apply_config()`** to reload values

## Frequently Asked Questions

### What is the default LLM provider in LogSentinelAI?

The default provider is **OpenAI** using the `gpt-4o-mini` model. This is defined in [`src/logsentinelai/core/config.py`](https://github.com/call518/logsentinelai/blob/main/src/logsentinelai/core/config.py) where `LLM_PROVIDER` defaults to `"openai"` and `LLM_MODEL_OPENAI` defaults to `"gpt-4o-mini"` when environment variables are unset.

### Can I switch LLM providers without restarting my application?

Yes. Update your `.env` file or environment variables, then invoke **`apply_config()`** from [`src/logsentinelai/core/config.py`](https://github.com/call518/logsentinelai/blob/main/src/logsentinelai/core/config.py) to reload configuration values. However, existing model client instances created via `initialize_llm_model` must be recreated to use the new provider settings, as the factory binds the client to specific endpoints at initialization.

### Does LogSentinelAI support custom local models beyond the defaults?

Absolutely. Set `LLM_PROVIDER` to `ollama` or `vllm`, then override the model name using `LLM_MODEL_OLLAMA` or `LLM_MODEL_VLLM`. For example, setting `LLM_MODEL_OLLAMA=mixtral:latest` routes requests to your local Mixtral instance without requiring code modifications, provided your Ollama server hosts that model.

### Why does the Gemini provider use the OpenAI client class?

LogSentinelAI leverages the **OpenAI SDK compatibility layer** offered by Google's Gemini API. This design choice allows the codebase to maintain a single client implementation (`openai.OpenAI`) while supporting Gemini's native models. The client points to `https://generativelanguage.googleapis.com/v1beta/openai/` and authenticates using `GEMINI_API_KEY`.