Supported LLM Providers in LogSentinelAI and How to Configure Them

LogSentinelAI supports four LLM backends—Ollama, vLLM, OpenAI, and Gemini—each configured via environment variables loaded by src/logsentinelai/core/config.py and instantiated through the factory function initialize_llm_model in src/logsentinelai/core/llm.py.

LogSentinelAI is an open-source log analysis framework that abstracts LLM interactions through a unified client interface. Understanding the supported LLM providers in LogSentinelAI and how to configure them allows you to route log analysis tasks to local models via Ollama or vLLM, or to cloud endpoints like OpenAI and Gemini without modifying application code.

Supported LLM Providers Overview

The initialize_llm_model function in src/logsentinelai/core/llm.py (lines 20-70) routes requests to four distinct backends. All providers use the OpenAI SDK client with customized base_url and authentication parameters.

Ollama (Local Execution)

Ollama enables on-premise model serving. The client wraps openai.OpenAI with a dummy API key and points to your local Ollama server.

• Default Model: qwen2.5-coder:3b
• Default Host: http://127.0.0.1:11434/v1
• Environment Variables: LLM_API_HOST_OLLAMA, LLM_MODEL_OLLAMA

vLLM (High-Throughput Serving)

vLLM supports optimized local or remote model hosting. Like Ollama, it uses the OpenAI client shim but targets vLLM inference endpoints.

• Default Model: Qwen/Qwen2.5-1.5B-Instruct
• Default Host: http://127.0.0.1:5000/v1
• Environment Variables: LLM_API_HOST_VLLM, LLM_MODEL_VLLM

OpenAI (Cloud API)

OpenAI connects to the official API using your account credentials. The SDK reads the secret directly from the environment.

• Default Model: gpt-4o-mini
• Default Host: https://api.openai.com/v1
• Required Secret: OPENAI_API_KEY

Gemini (Google Cloud)

Gemini routes through Google's Generative Language API using an OpenAI-compatible shim.

• Default Model: gemini-1.5-pro
• Default Host: https://generativelanguage.googleapis.com/v1beta/openai/
• Required Secret: GEMINI_API_KEY

Configuration Environment Variables

All provider settings are loaded at runtime from environment variables by the _load_values helper in src/logsentinelai/core/config.py (lines 61-73). The global provider selector defaults to openai if unspecified.

LLM_PROVIDER = os.getenv("LLM_PROVIDER", "openai")
LLM_MODELS = {
    "ollama": os.getenv("LLM_MODEL_OLLAMA", "qwen2.5-coder:3b"),
    "vllm":   os.getenv("LLM_MODEL_VLLM",   "Qwen/Qwen2.5-1.5B-Instruct"),
    "openai": os.getenv("LLM_MODEL_OPENAI", "gpt-4o-mini"),
    "gemini": os.getenv("LLM_MODEL_GEMINI", "gemini-1.5-pro"),
}
LLM_API_HOSTS = {
    "ollama": os.getenv("LLM_API_HOST_OLLAMA", "http://127.0.0.1:11434/v1"),
    "vllm":   os.getenv("LLM_API_HOST_VLLM",   "http://127.0.0.1:5000/v1"),
    "openai": os.getenv("LLM_API_HOST_OPENAI", "https://api.openai.com/v1"),
    "gemini": os.getenv("LLM_API_HOST_GEMINI", "https://generativelanguage.googleapis.com/v1beta/openai/"),
}

Step-by-Step Configuration Guide

1. Copy the environment template from the repository root:

   cp .env.template .env

2. Set the global provider (optional—defaults to openai):

   LLM_PROVIDER=ollama          # or vllm, openai, gemini
   

3. Override model names (optional):

   LLM_MODEL_OLLAMA=qwen2.5-coder:3b
   LLM_MODEL_VLLM=Qwen/Qwen2.5-1.5B-Instruct
   LLM_MODEL_OPENAI=gpt-4o-mini
   LLM_MODEL_GEMINI=gemini-1.5-pro

4. Configure endpoint URLs for self-hosted instances:

   LLM_API_HOST_OLLAMA=http://127.0.0.1:11434/v1
   LLM_API_HOST_VLLM=http://127.0.0.1:5000/v1

5. Provide authentication tokens for cloud providers:

   OPENAI_API_KEY=sk-...
   GEMINI_API_KEY=AIza...

6. Reload configuration at runtime after changes:

   from logsentinelai.core.config import apply_config
   apply_config()

Initializing and Using LLM Clients in Code

The factory function initialize_llm_model instantiates the correct client based on the provider string. Use generate_with_model for unified inference across all backends, including special post-processing for Gemini responses.

from logsentinelai.core.llm import initialize_llm_model, generate_with_model
from logsentinelai.core.config import LLM_PROVIDER, LLM_MODELS
from pydantic import BaseModel

# 1. Select provider (overrides .env if needed)

provider = "gemini"                       # could be "ollama", "vllm", "openai", "gemini"

model_name = LLM_MODELS[provider]

# 2. Build the model object

model = initialize_llm_model(llm_provider=provider, llm_model_name=model_name)

# 3. Define a Pydantic schema for structured output

class IssueReport(BaseModel):
    level: str
    description: str
    timestamp: str

# 4. Send a prompt and get validated JSON

prompt = "Summarize the most critical error in the following log snippet..."
json_result = generate_with_model(model, prompt, IssueReport, llm_provider=provider)

print(json_result)   # → validated JSON string

This implementation works for any of the four providers because generate_with_model adapts the call internally—Gemini receives markdown stripping and validation, while Ollama, vLLM, and OpenAI use the standard outlines wrapper.

Provider-Specific Implementation Details

Client Instantiation Logic

In src/logsentinelai/core/llm.py, the initialize_llm_model function (lines 20-70) matches the provider string and constructs the appropriate openai.OpenAI instance. Ollama and vLLM use dummy API keys with custom base_url parameters, while OpenAI and Gemini pass real authentication headers via environment variables.

Response Handling Differences

While Ollama, vLLM, and OpenAI use standard JSON schema enforcement, Gemini requires special post-processing. The generate_with_model function strips markdown formatting from Gemini responses before Pydantic validation, ensuring consistent output structure across all providers.

Summary

• LogSentinelAI supports four LLM providers: Ollama, vLLM, OpenAI, and Gemini
• Configuration occurs through environment variables defined in src/logsentinelai/core/config.py
• Local providers require host URL configuration, while cloud providers need API keys (OPENAI_API_KEY or GEMINI_API_KEY)
• The initialize_llm_model factory in src/logsentinelai/core/llm.py abstracts client creation and routing
• Runtime configuration changes require calling apply_config() to reload values

Frequently Asked Questions

What is the default LLM provider in LogSentinelAI?

The default provider is OpenAI using the gpt-4o-mini model. This is defined in src/logsentinelai/core/config.py where LLM_PROVIDER defaults to "openai" and LLM_MODEL_OPENAI defaults to "gpt-4o-mini" when environment variables are unset.

Can I switch LLM providers without restarting my application?

Yes. Update your .env file or environment variables, then invoke apply_config() from src/logsentinelai/core/config.py to reload configuration values. However, existing model client instances created via initialize_llm_model must be recreated to use the new provider settings, as the factory binds the client to specific endpoints at initialization.

Does LogSentinelAI support custom local models beyond the defaults?

Absolutely. Set LLM_PROVIDER to ollama or vllm, then override the model name using LLM_MODEL_OLLAMA or LLM_MODEL_VLLM. For example, setting LLM_MODEL_OLLAMA=mixtral:latest routes requests to your local Mixtral instance without requiring code modifications, provided your Ollama server hosts that model.

Why does the Gemini provider use the OpenAI client class?

LogSentinelAI leverages the OpenAI SDK compatibility layer offered by Google's Gemini API. This design choice allows the codebase to maintain a single client implementation (openai.OpenAI) while supporting Gemini's native models. The client points to https://generativelanguage.googleapis.com/v1beta/openai/ and authenticates using GEMINI_API_KEY.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}