How to Onboard and Configure LLM Providers in OpenSRE: A Complete Guide

You onboard LLM providers in OpenSRE by setting the LLM_PROVIDER environment variable, supplying API credentials via environment variables or the OS keyring, and calling get_llm_for_reasoning() or get_llm_for_tools() to receive a ready-to-use client.

OpenSRE from Tracer-Cloud unifies multiple large language model backends behind a single provider-agnostic interface. Learning how to onboard and configure LLM providers in OpenSRE lets you switch between Anthropic, OpenAI, AWS Bedrock, or local Ollama instances without changing your application code.

Understanding the LLM Provider Architecture in OpenSRE

OpenSRE centralizes provider configuration in app/config.py through the LLMSettings class. This class validates provider names, resolves API keys, and loads model-specific overrides. The actual client instantiation happens in app/services/llm_client.py, where the _create_llm_client() factory maps provider strings to concrete wrapper classes like OpenAILLMClient, BedrockLLMClient, or OllamaLLMClient.

Step-by-Step: How to Onboard and Configure LLM Providers in OpenSRE

Step 1: Select Your Provider via Environment Variables

Set the LLM_PROVIDER variable to one of the supported literals: anthropic, openai, openrouter, gemini, nvidia, ollama, bedrock, or minimax.

export LLM_PROVIDER=anthropic

The _normalize_provider method in app/config.py (lines 58–66) validates this value against the whitelist and raises a descriptive error if you provide an unsupported provider.

Step 2: Securely Configure API Credentials

OpenSRE resolves credentials through the resolve_llm_api_key helper in app/llm_credentials.py (lines 19–27). It checks environment variables first, then falls back to the OS keyring.

Option A: Environment variable

export ANTHROPIC_API_KEY=sk-ant-...

Option B: OS keyring (more secure)

from app.llm_credentials import save_llm_api_key

save_llm_api_key("ANTHROPIC_API_KEY", "sk-ant-...")

The save_llm_api_key function (lines 37–49 in app/llm_credentials.py) uses the system keyring backend to encrypt and store the secret.

Step 3: Override Default Model Settings (Optional)

You can customize model IDs and token limits without touching code. In app/config.py (lines 190–226), LLMSettings.from_env() reads provider-specific overrides:

export ANTHROPIC_REASONING_MODEL=claude-3-opus-20240229
export ANTHROPIC_TOOLCALL_MODEL=claude-3-haiku-20240307
export MAX_TOKENS=4096

Step 4: Initialize the LLM Client

Call the factory functions in app/services/llm_client.py to receive a configured client:

from app.services.llm_client import get_llm_for_reasoning, get_llm_for_tools

# High-cost, complex reasoning tasks

reasoning_llm = get_llm_for_reasoning()

# Lightweight routing and tool calls

tool_llm = get_llm_for_tools()

The _create_llm_client factory (lines 52–124) instantiates the correct wrapper class—such as OpenAILLMClient, BedrockLLMClient, or OllamaLLMClient—based on your LLM_PROVIDER setting.

Configuration Examples for Popular LLM Providers

OpenAI Configuration

export LLM_PROVIDER=openai
export OPENAI_API_KEY=sk-...
export OPENAI_REASONING_MODEL=gpt-4o
export OPENAI_TOOLCALL_MODEL=gpt-4o-mini

from app.services.llm_client import get_llm_for_reasoning

llm = get_llm_for_reasoning()
response = llm.invoke([{"role": "user", "content": "Analyze error rate trends"}])

Anthropic (Default) Configuration

Anthropic is the default provider. If you have set ANTHROPIC_API_KEY in your environment or keyring, no additional configuration is required.

from app.services.llm_client import get_llm_for_tools

# Uses Claude 3 Haiku or your configured ANTHROPIC_TOOLCALL_MODEL

llm = get_llm_for_tools()

Local Ollama Setup

For air-gapped or cost-sensitive environments, use local Ollama instances:

export LLM_PROVIDER=ollama
export OLLAMA_HOST=http://localhost:11434
export OLLAMA_MODEL=llama3.2

from app.services.llm_client import get_llm_for_reasoning
from app.cli.local_llm.ollama import pull_model

# Ensure model is cached locally

pull_model("llama3.2", console=None)

llm = get_llm_for_reasoning()
print(llm.invoke("Explain the four golden signals of SRE").content)

The Ollama branch in _create_llm_client (lines 124–135) handles model tag normalization and bypasses API key resolution since Ollama runs locally.

AWS Bedrock and Other Providers

AWS Bedrock follows the same pattern but requires AWS credentials (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_REGION) instead of a provider-specific API key:

export LLM_PROVIDER=bedrock
export AWS_REGION=us-west-2

The BedrockLLMClient class in app/services/llm_client.py handles the boto3 session initialization and model invocation.

Summary

• Set LLM_PROVIDER to one of the eight supported backends (anthropic, openai, bedrock, ollama, etc.) to select your LLM backend.
• Secure credentials using environment variables or the OS keyring via save_llm_api_key; resolution happens automatically in resolve_llm_api_key.
• Customize models by exporting provider-specific variables like ANTHROPIC_REASONING_MODEL or MAX_TOKENS before calling LLMSettings.from_env().
• Initialize clients using get_llm_for_reasoning() or get_llm_for_tools() from app/services/llm_client.py to receive a configured, ready-to-use wrapper.

Frequently Asked Questions

What LLM providers does OpenSRE support?

OpenSRE supports eight providers: Anthropic, OpenAI, OpenRouter, Google Gemini, Nvidia, Ollama, AWS Bedrock, and MiniMax. The whitelist is enforced in app/config.py inside the _normalize_provider method, which raises a validation error if you specify an unsupported provider string.

How does OpenSRE handle API key security?

OpenSRE uses a two-tier resolution strategy defined in app/llm_credentials.py. The resolve_llm_api_key function first checks for an environment variable (e.g., ANTHROPIC_API_KEY), then falls back to the OS keyring. You can store keys securely using the save_llm_api_key helper, which encrypts credentials using the system keyring backend, keeping secrets out of shell history and environment dumps.

Can I use a local LLM with OpenSRE?

Yes. Set LLM_PROVIDER=ollama and optionally specify OLLAMA_HOST and OLLAMA_MODEL. The _create_llm_client factory in app/services/llm_client.py instantiates an Ollama-specific client that bypasses API key checks. Use the pull_model helper from app/cli/local_llm/ollama.py to ensure the model is cached locally before invoking the client.

How do I switch between different LLM providers?

Switching providers requires only changing the LLM_PROVIDER environment variable and ensuring the corresponding API credentials are available. OpenSRE’s factory pattern in app/services/llm_client.py handles the instantiation logic; you do not need to modify code. Simply export the new provider name (e.g., export LLM_PROVIDER=bedrock), set the appropriate credentials (e.g., AWS_ACCESS_KEY_ID), and call get_llm_for_reasoning() to receive the new client.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}