# How to Configure Multiple LLM Providers in Embabel: OpenAI, Anthropic, and Ollama

> Learn to configure multiple LLM providers like OpenAI, Anthropic, and Ollama in Embabel. Map model names and inject LlmService beans for seamless integration.

- Repository: [Embabel/embabel-agent](https://github.com/embabel/embabel-agent)
- Tags: how-to-guide
- Published: 2026-08-08

---

**You can configure multiple LLM providers in Embabel by including provider-specific starter dependencies, mapping symbolic model names to concrete providers under `embabel.models.llms`, and injecting specific `LlmService` beans using Spring's `@Qualifier` annotation or the `LlmServiceRegistry`.**

The `embabel/embabel-agent` repository provides a provider-agnostic abstraction layer that unifies disparate LLM vendors behind a common interface. By leveraging Spring Boot auto-configuration, you can simultaneously integrate OpenAI, Anthropic, Ollama, and OCI GenAI within the same application, switching between them via configuration or programmatic selection.

## Understanding the Provider-Agnostic Architecture

Embabel’s architecture centers on the **`LlmService`** interface, which abstracts provider-specific implementations. Each LLM vendor ships an **auto-configuration class** (e.g., `AgentOpenAiAutoConfiguration`) that registers beans implementing this interface. 

Key components include:

- **LLM Model Registry**: The property namespace `embabel.models.llms` maps symbolic names (like `best` or `cheap`) to provider-specific model identifiers (like `gpt-4o` or `claude-3.5-sonnet`).
- **Default LLM**: The property `embabel.models.default-llm` specifies which symbolic name to use when no explicit qualifier is provided.
- **LlmServiceRegistry**: A programmatic registry that exposes a `get(String modelKey)` method to fetch the appropriate service bean at runtime.

## Step-by-Step Configuration Guide

### 1. Add Provider Dependencies

Include the starter modules for each LLM provider you need in your [`pom.xml`](https://github.com/embabel/embabel-agent/blob/main/pom.xml). Each starter pulls in the necessary auto-configuration classes.

```xml
<!-- OpenAI -->
<dependency>
    <groupId>com.embabel</groupId>
    <artifactId>embabel-agent-starter-openai</artifactId>
    <version>${embabel.version}</version>
</dependency>

<!-- Anthropic -->
<dependency>
    <groupId>com.embabel</groupId>
    <artifactId>embabel-agent-starter-anthropic</artifactId>
    <version>${embabel.version}</version>
</dependency>

<!-- Ollama -->
<dependency>
    <groupId>com.embabel</groupId>
    <artifactId>embabel-agent-starter-ollama</artifactId>
    <version>${embabel.version}</version>
</dependency>

```

### 2. Configure Model Mappings

Define your model aliases and default provider in [`application.yml`](https://github.com/embabel/embabel-agent/blob/main/application.yml). Each entry under `embabel.models.llms.<name>` corresponds to a provider-specific model identifier.

```yaml
embabel:
  models:
    default-llm: best
    llms:
      best: gpt-4o
      cheap: gpt-4o-mini
      anthropic: claude-3.5-sonnet
      ollama: llama3-70b-instruct

```

### 3. Inject Specific LLM Beans

To use multiple providers simultaneously within the same component, inject the desired `LlmService` bean using the `@Qualifier` annotation. The qualifier value must match the symbolic name defined in your configuration.

```java
@Component
public class MultiLlmProcessor {

    private final LlmService openAiService;
    private final LlmService anthropicService;

    public MultiLlmProcessor(
            @Qualifier("best") LlmService openAiService,
            @Qualifier("anthropic") LlmService anthropicService) {
        this.openAiService = openAiService;
        this.anthropicService = anthropicService;
    }

    public Mono<String> answerWithOpenAi(String prompt) {
        return openAiService.createObject(prompt, String.class);
    }

    public Mono<String> answerWithAnthropic(String prompt) {
        return anthropicService.createObject(prompt, String.class);
    }
}

```

### 4. Dynamic Provider Selection at Runtime

For scenarios requiring runtime provider selection, use the `LlmServiceRegistry` to fetch beans dynamically by model key.

```java
@Component
public class ChatOrchestrator {

    private final LlmServiceRegistry registry;

    public ChatOrchestrator(LlmServiceRegistry registry) {
        this.registry = registry;
    }

    public Mono<String> chat(String providerKey, String prompt) {
        LlmService service = registry.get(providerKey);
        return service.createObject(prompt, String.class);
    }
}

```

## Provider-Specific Auto-Configuration Classes

Each provider is activated by a distinct auto-configuration class located in the `embabel-agent-autoconfigure` module. These classes read the `embabel.models.llms` entries and instantiate the appropriate `LlmService` implementation.

| Provider | Auto-Configuration Class | Source Path |
|----------|-------------------------|-------------|
| **OpenAI** | `AgentOpenAiAutoConfiguration` | [`embabel-agent-autoconfigure/models/embabel-agent-openai-autoconfigure/src/main/java/com/embabel/agent/autoconfigure/models/openai/AgentOpenAiAutoConfiguration.java`](https://github.com/embabel/embabel-agent/blob/main/embabel-agent-autoconfigure/models/embabel-agent-openai-autoconfigure/src/main/java/com/embabel/agent/autoconfigure/models/openai/AgentOpenAiAutoConfiguration.java) |
| **Anthropic** | `AgentAnthropicAutoConfiguration` | [`src/main/java/com/embabel/agent/autoconfigure/models/anthropic/AgentAnthropicAutoConfiguration.java`](https://github.com/embabel/embabel-agent/blob/main/src/main/java/com/embabel/agent/autoconfigure/models/anthropic/AgentAnthropicAutoConfiguration.java) |
| **Ollama** | `AgentOllamaAutoConfiguration` | [`src/main/java/com/embabel/agent/autoconfigure/models/ollama/AgentOllamaAutoConfiguration.java`](https://github.com/embabel/embabel-agent/blob/main/src/main/java/com/embabel/agent/autoconfigure/models/ollama/AgentOllamaAutoConfiguration.java) |
| **OCI GenAI** | `OciGenAiEnvironmentPostProcessor` | [`embabel-agent-autoconfigure/models/embabel-agent-oci-genai-autoconfigure/src/main/java/com/embabel/agent/autoconfigure/models/ocigenai/OciGenAiEnvironmentPostProcessor.java`](https://github.com/embabel/embabel-agent/blob/main/embabel-agent-autoconfigure/models/embabel-agent-oci-genai-autoconfigure/src/main/java/com/embabel/agent/autoconfigure/models/ocigenai/OciGenAiEnvironmentPostProcessor.java) |

Providers are only activated when their auto-configuration class is present on the classpath, allowing you to control availability via dependencies alone.

## Provider-Specific Properties

Configure API keys, timeouts, and model parameters under provider-specific prefixes. These properties are handled automatically by the respective auto-configuration classes.

```yaml
embabel:
  openai:
    api-key: ${OPENAI_API_KEY}
    timeout: 30s
  
  anthropic:
    api-key: ${ANTHROPIC_API_KEY}
    temperature: 0.7
  
  ollama:
    base-url: http://localhost:11434

```

## Observability and Tracing

When you configure multiple LLM providers, the observation subsystem automatically tags spans with `embabel.llm.model` based on the bean name. This enables unified metrics and tracing across all providers. The relevant conventions are implemented in:

- `EmbabelLlmObservationConvention` ([`embabel-agent-observability/src/main/java/com/embabel/agent/observability/tracing/EmbabelLlmObservationConvention.java`](https://github.com/embabel/embabel-agent/blob/main/embabel-agent-observability/src/main/java/com/embabel/agent/observability/tracing/EmbabelLlmObservationConvention.java))
- `EmbabelSpanEventListener` ([`embabel-agent-observability/src/main/java/com/embabel/agent/observability/tracing/EmbabelSpanEventListener.java`](https://github.com/embabel/embabel-agent/blob/main/embabel-agent-observability/src/main/java/com/embabel/agent/observability/tracing/EmbabelSpanEventListener.java))

## Summary

- **Add starter dependencies** for each provider you want to use (OpenAI, Anthropic, Ollama, etc.).
- **Map symbolic names** to concrete model IDs under `embabel.models.llms` in your YAML configuration.
- **Set a default provider** using `embabel.models.default-llm` for fallback injection.
- **Inject specific beans** using `@Qualifier` with the symbolic model name, or use `LlmServiceRegistry` for dynamic lookup.
- **Configure provider-specific settings** (API keys, timeouts) under dedicated property namespaces like `embabel.openai` or `embabel.anthropic`.

## Frequently Asked Questions

### How do I add a custom LLM provider to Embabel?

You can add a custom provider by creating a new auto-configuration class annotated with `@Configuration` that registers a bean implementing `LlmService`. Name the bean after the model identifier and place the class in your classpath. The `AgentZaiAutoConfiguration` class in the source code serves as an example of this pattern for the Z-AI provider.

### Can I use the same model name for different providers?

No, the symbolic names under `embabel.models.llms` must be unique as they serve as bean qualifiers. If you need to reference the same conceptual model across providers, use distinct keys (e.g., `gpt-best` and `claude-best`) and map them accordingly in your configuration.

### How does Embabel handle API keys for different providers?

Each provider's auto-configuration class reads API keys from its dedicated property namespace. For example, `AgentOpenAiAutoConfiguration` consumes `embabel.openai.api-key`, while `AgentAnthropicAutoConfiguration` consumes `embabel.anthropic.api-key`. These are resolved independently, allowing you to securely manage credentials for multiple providers simultaneously.

### What happens if the default LLM provider is unavailable?

If the default LLM provider specified in `embabel.models.default-llm` is unavailable, the application will throw a standard Spring `NoSuchBeanDefinitionException` at runtime if no fallback is configured. To handle provider failures gracefully, implement circuit breakers or use the `LlmServiceRegistry` to programmatically fall back to an alternative provider when a specific bean is unavailable.