OpenSRE LLM Providers: Which AI Models Can OpenSRE Integrate With?

OpenSRE supports eight LLM providers including Anthropic, OpenAI, Google Gemini, AWS Bedrock, NVIDIA, Ollama, OpenRouter, and Minimax, configurable via the LLM_PROVIDER environment variable.

OpenSRE is an open-source Site Reliability Engineering platform maintained by Tracer-Cloud that leverages AI for infrastructure monitoring and incident response. The platform's multi-provider architecture allows teams to integrate with various large language model providers based on their specific security, performance, and cost requirements.

Supported LLM Providers in OpenSRE

The complete list of supported providers is defined by the LLMProvider type in app/config.py. According to the OpenSRE source code, the following providers are available:

• anthropic: Anthropic models (e.g., Claude)
• openai: OpenAI models (e.g., GPT-4, GPT-3.5)
• openrouter: Any model reachable through the OpenRouter gateway
• gemini: Google Gemini models
• nvidia: NVIDIA AI Foundation models
• ollama: Local Ollama server models
• bedrock: AWS Bedrock (Anthropic via IAM-authenticated Bedrock)
• minimax: Minimax AI models

How LLM Provider Configuration Works

OpenSRE implements a type-safe configuration system that validates provider selection at both compile-time and runtime.

The LLMProvider Type Definition

In app/config.py, the LLMProvider literal type enumerates all valid provider strings. This strict typing ensures that only supported providers can be configured, preventing runtime errors from invalid provider names.

Environment Variable Resolution

Users select providers via the LLM_PROVIDER environment variable, which defaults to anthropic when not specified. The resolve_llm_provider helper function in app/utils/cfg_helpers.py handles the environment variable parsing and validation.

Implementing Multi-Provider Support

The LLM Client Service

The app/services/llm_client.py file contains the main LLMClient class that dynamically selects the appropriate SDK based on the configured provider. As implemented in Tracer-Cloud/opensre, this client initializes provider-specific settings through the LLMSettings configuration object and manages authentication credentials for each service.

Configuration Helpers

The app/utils/cfg_helpers.py module provides utilities for reading and validating the LLM_PROVIDER value from the environment, ensuring type safety before the LLM client instantiation occurs.

Practical Configuration Examples

Setting Provider via Environment Variable

import os

# Choose the provider you want to use

os.environ["LLM_PROVIDER"] = "openai"      # or "anthropic", "gemini", etc.

# The OpenSRE LLM client will read this value automatically

from app.services.llm_client import LLMClient

client = LLMClient()
print(client.provider)   # → "openai"

Direct Provider Instantiation

from app.services.llm_client import LLMClient, LLMSettings

settings = LLMSettings(
    provider="bedrock",
    bedrock_reasoning_model="anthropic.claude-v2",
    bedrock_toolcall_model="anthropic.claude-v2"
)

client = LLMClient(settings=settings)
print(client.provider)   # → "bedrock"

Listing Supported Providers Programmatically

from app.config import LLMProvider

def list_supported_providers() -> list[str]:
    # The Literal type resolves to a tuple of strings at runtime

    return list(LLMProvider.__args__)   # type: ignore[attr-defined]

print(list_supported_providers())

# → ['anthropic', 'openai', 'openrouter', 'gemini', 'nvidia', 'ollama', 'bedrock', 'minimax']

Testing Provider Integration

The test suite in tests/nodes/test_chat_provider_awareness.py validates that the correct provider-specific chat model loads based on the configuration. These unit tests confirm that provider switching works correctly across the application and that each provider's specific authentication and API requirements are properly handled.

Summary

• OpenSRE supports eight LLM providers: Anthropic, OpenAI, OpenRouter, Google Gemini, NVIDIA, Ollama, AWS Bedrock, and Minimax.
• Provider selection is controlled via the LLM_PROVIDER environment variable, defaulting to anthropic when not specified.
• The LLMProvider literal type in app/config.py provides compile-time validation of supported providers.
• Runtime configuration is handled by LLMClient in app/services/llm_client.py with helper functions in app/utils/cfg_helpers.py.

Frequently Asked Questions

What is the default LLM provider in OpenSRE?

The default provider is anthropic. When the LLM_PROVIDER environment variable is not set, OpenSRE automatically configures itself to use Anthropic's Claude models according to the default configuration values in app/config.py.

Can I use local LLM models with OpenSRE?

Yes. OpenSRE supports Ollama as a first-class provider, allowing you to run local models on your own infrastructure without sending data to external APIs. Configure this by setting LLM_PROVIDER=ollama in your environment variables.

How does OpenSRE validate LLM provider configuration?

Provider validation occurs through the LLMProvider literal type definition in app/config.py. This Python Literal type restricts valid values to the eight supported providers, preventing configuration errors at initialization time when the LLMClient attempts to load the appropriate SDK.

Can I switch between cloud and local providers dynamically?

Yes. Since OpenSRE reads the LLM_PROVIDER environment variable at startup, you can switch between cloud providers like AWS Bedrock or OpenAI and local options like Ollama by modifying this variable before launching the application. The LLMClient class in app/services/llm_client.py handles the SDK selection and authentication automatically based on the current configuration.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}