# How to Onboard and Configure LLM Providers in OpenSRE: A Complete Guide

> Learn to onboard and configure LLM providers in OpenSRE with this comprehensive guide. Easily integrate LLM clients for reasoning and tool use. Get started today!

- Repository: [Tracer/opensre](https://github.com/Tracer-Cloud/opensre)
- Tags: how-to-guide
- Published: 2026-04-18

---

**You onboard LLM providers in OpenSRE by setting the `LLM_PROVIDER` environment variable, supplying API credentials via environment variables or the OS keyring, and calling `get_llm_for_reasoning()` or `get_llm_for_tools()` to receive a ready-to-use client.**

OpenSRE from Tracer-Cloud unifies multiple large language model backends behind a single provider-agnostic interface. Learning how to onboard and configure LLM providers in OpenSRE lets you switch between Anthropic, OpenAI, AWS Bedrock, or local Ollama instances without changing your application code.

## Understanding the LLM Provider Architecture in OpenSRE

OpenSRE centralizes provider configuration in **[`app/config.py`](https://github.com/Tracer-Cloud/opensre/blob/main/app/config.py)** through the `LLMSettings` class. This class validates provider names, resolves API keys, and loads model-specific overrides. The actual client instantiation happens in **[`app/services/llm_client.py`](https://github.com/Tracer-Cloud/opensre/blob/main/app/services/llm_client.py)**, where the `_create_llm_client()` factory maps provider strings to concrete wrapper classes like `OpenAILLMClient`, `BedrockLLMClient`, or `OllamaLLMClient`.

## Step-by-Step: How to Onboard and Configure LLM Providers in OpenSRE

### Step 1: Select Your Provider via Environment Variables

Set the `LLM_PROVIDER` variable to one of the supported literals: `anthropic`, `openai`, `openrouter`, `gemini`, `nvidia`, `ollama`, `bedrock`, or `minimax`.

```bash
export LLM_PROVIDER=anthropic

```

The `_normalize_provider` method in [`app/config.py`](https://github.com/Tracer-Cloud/opensre/blob/main/app/config.py) (lines 58–66) validates this value against the whitelist and raises a descriptive error if you provide an unsupported provider.

### Step 2: Securely Configure API Credentials

OpenSRE resolves credentials through the `resolve_llm_api_key` helper in **[`app/llm_credentials.py`](https://github.com/Tracer-Cloud/opensre/blob/main/app/llm_credentials.py)** (lines 19–27). It checks environment variables first, then falls back to the OS keyring.

**Option A: Environment variable**

```bash
export ANTHROPIC_API_KEY=sk-ant-...

```

**Option B: OS keyring (more secure)**

```python
from app.llm_credentials import save_llm_api_key

save_llm_api_key("ANTHROPIC_API_KEY", "sk-ant-...")

```

The `save_llm_api_key` function (lines 37–49 in [`app/llm_credentials.py`](https://github.com/Tracer-Cloud/opensre/blob/main/app/llm_credentials.py)) uses the system keyring backend to encrypt and store the secret.

### Step 3: Override Default Model Settings (Optional)

You can customize model IDs and token limits without touching code. In [`app/config.py`](https://github.com/Tracer-Cloud/opensre/blob/main/app/config.py) (lines 190–226), `LLMSettings.from_env()` reads provider-specific overrides:

```bash
export ANTHROPIC_REASONING_MODEL=claude-3-opus-20240229
export ANTHROPIC_TOOLCALL_MODEL=claude-3-haiku-20240307
export MAX_TOKENS=4096

```

### Step 4: Initialize the LLM Client

Call the factory functions in **[`app/services/llm_client.py`](https://github.com/Tracer-Cloud/opensre/blob/main/app/services/llm_client.py)** to receive a configured client:

```python
from app.services.llm_client import get_llm_for_reasoning, get_llm_for_tools

# High-cost, complex reasoning tasks

reasoning_llm = get_llm_for_reasoning()

# Lightweight routing and tool calls

tool_llm = get_llm_for_tools()

```

The `_create_llm_client` factory (lines 52–124) instantiates the correct wrapper class—such as `OpenAILLMClient`, `BedrockLLMClient`, or `OllamaLLMClient`—based on your `LLM_PROVIDER` setting.

## Configuration Examples for Popular LLM Providers

### OpenAI Configuration

```bash
export LLM_PROVIDER=openai
export OPENAI_API_KEY=sk-...
export OPENAI_REASONING_MODEL=gpt-4o
export OPENAI_TOOLCALL_MODEL=gpt-4o-mini

```

```python
from app.services.llm_client import get_llm_for_reasoning

llm = get_llm_for_reasoning()
response = llm.invoke([{"role": "user", "content": "Analyze error rate trends"}])

```

### Anthropic (Default) Configuration

Anthropic is the default provider. If you have set `ANTHROPIC_API_KEY` in your environment or keyring, no additional configuration is required.

```python
from app.services.llm_client import get_llm_for_tools

# Uses Claude 3 Haiku or your configured ANTHROPIC_TOOLCALL_MODEL

llm = get_llm_for_tools()

```

### Local Ollama Setup

For air-gapped or cost-sensitive environments, use local Ollama instances:

```bash
export LLM_PROVIDER=ollama
export OLLAMA_HOST=http://localhost:11434
export OLLAMA_MODEL=llama3.2

```

```python
from app.services.llm_client import get_llm_for_reasoning
from app.cli.local_llm.ollama import pull_model

# Ensure model is cached locally

pull_model("llama3.2", console=None)

llm = get_llm_for_reasoning()
print(llm.invoke("Explain the four golden signals of SRE").content)

```

The Ollama branch in `_create_llm_client` (lines 124–135) handles model tag normalization and bypasses API key resolution since Ollama runs locally.

### AWS Bedrock and Other Providers

AWS Bedrock follows the same pattern but requires AWS credentials (`AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, `AWS_REGION`) instead of a provider-specific API key:

```bash
export LLM_PROVIDER=bedrock
export AWS_REGION=us-west-2

```

The `BedrockLLMClient` class in [`app/services/llm_client.py`](https://github.com/Tracer-Cloud/opensre/blob/main/app/services/llm_client.py) handles the boto3 session initialization and model invocation.

## Summary

- **Set `LLM_PROVIDER`** to one of the eight supported backends (`anthropic`, `openai`, `bedrock`, `ollama`, etc.) to select your LLM backend.
- **Secure credentials** using environment variables or the OS keyring via `save_llm_api_key`; resolution happens automatically in `resolve_llm_api_key`.
- **Customize models** by exporting provider-specific variables like `ANTHROPIC_REASONING_MODEL` or `MAX_TOKENS` before calling `LLMSettings.from_env()`.
- **Initialize clients** using `get_llm_for_reasoning()` or `get_llm_for_tools()` from [`app/services/llm_client.py`](https://github.com/Tracer-Cloud/opensre/blob/main/app/services/llm_client.py) to receive a configured, ready-to-use wrapper.

## Frequently Asked Questions

### What LLM providers does OpenSRE support?

OpenSRE supports eight providers: **Anthropic**, **OpenAI**, **OpenRouter**, **Google Gemini**, **Nvidia**, **Ollama**, **AWS Bedrock**, and **MiniMax**. The whitelist is enforced in [`app/config.py`](https://github.com/Tracer-Cloud/opensre/blob/main/app/config.py) inside the `_normalize_provider` method, which raises a validation error if you specify an unsupported provider string.

### How does OpenSRE handle API key security?

OpenSRE uses a two-tier resolution strategy defined in [`app/llm_credentials.py`](https://github.com/Tracer-Cloud/opensre/blob/main/app/llm_credentials.py). The `resolve_llm_api_key` function first checks for an environment variable (e.g., `ANTHROPIC_API_KEY`), then falls back to the OS keyring. You can store keys securely using the `save_llm_api_key` helper, which encrypts credentials using the system keyring backend, keeping secrets out of shell history and environment dumps.

### Can I use a local LLM with OpenSRE?

Yes. Set `LLM_PROVIDER=ollama` and optionally specify `OLLAMA_HOST` and `OLLAMA_MODEL`. The `_create_llm_client` factory in [`app/services/llm_client.py`](https://github.com/Tracer-Cloud/opensre/blob/main/app/services/llm_client.py) instantiates an Ollama-specific client that bypasses API key checks. Use the `pull_model` helper from [`app/cli/local_llm/ollama.py`](https://github.com/Tracer-Cloud/opensre/blob/main/app/cli/local_llm/ollama.py) to ensure the model is cached locally before invoking the client.

### How do I switch between different LLM providers?

Switching providers requires only changing the `LLM_PROVIDER` environment variable and ensuring the corresponding API credentials are available. OpenSRE’s factory pattern in [`app/services/llm_client.py`](https://github.com/Tracer-Cloud/opensre/blob/main/app/services/llm_client.py) handles the instantiation logic; you do not need to modify code. Simply export the new provider name (e.g., `export LLM_PROVIDER=bedrock`), set the appropriate credentials (e.g., `AWS_ACCESS_KEY_ID`), and call `get_llm_for_reasoning()` to receive the new client.