# How to Configure Multi-LLM Support in Embabel

> Configure multi-LLM support in Embabel using LlmOptions. Route LLMs by name, role, or fallback lists for flexible runtime control. Simplify your AI agent development.

- Repository: [Embabel/embabel-agent](https://github.com/embabel/embabel-agent)
- Tags: how-to-guide
- Published: 2026-08-08

---

**Embabel enables multi-LLM support through a model-agnostic `LlmOptions` class that pairs hyper-parameters with pluggable `ModelSelectionCriteria`, allowing runtime routing by name, role, or fallback lists.**

The `embabel/embabel-agent` repository provides a Spring Boot-based framework for building AI agents that can leverage multiple large language models simultaneously. By abstracting provider details behind a unified configuration layer, Embabel lets you define OpenAI, Anthropic, Azure, and other LLMs side-by-side, then route requests to the optimal model for each specific task.

## Understanding Embabel's Multi-LLM Architecture

Embabel's architecture decouples LLM capability definition from provider implementation. The framework discovers provider-specific beans at startup and resolves them at runtime based on selection criteria you define.

### The LlmOptions Class

At the center of multi-LLM configuration is `LlmOptions`, located in [`embabel-agent-common/embabel-agent-ai/src/main/kotlin/com/embabel/common/ai/model/LlmOptions.kt`](https://github.com/embabel/embabel-agent/blob/main/embabel-agent-common/embabel-agent-ai/src/main/kotlin/com/embabel/common/ai/model/LlmOptions.kt). This data class encapsulates:

- **Hyper-parameters**: Temperature, model name, API credentials
- **Selection criteria**: How the framework should choose the concrete provider
- **Extensions**: Provider-specific flags (e.g., Anthropic caching)

The `criteria` property (lines 38-52) computes the final `ModelSelectionCriteria` based on your configuration, determining which provider bean handles the request.

### Model Selection Criteria

Embabel implements several concrete selection strategies in the `ModelSelectionCriteria` hierarchy:

- **By Name**: Explicitly select a configured model (e.g., "gpt-4")
- **By Role**: Map functional roles (e.g., "summarizer") to specific models
- **Fallback-by-Name**: Try models in priority order until one succeeds
- **Platform Default**: Use the globally configured default LLM

Provider modules register their beans via factories like [`OpenAiCompatibleModelFactory.kt`](https://github.com/embabel/embabel-agent/blob/main/OpenAiCompatibleModelFactory.kt) in the `embabel-agent-openai` module, making them available to the selection engine.

## Configuring Multiple LLM Providers in application.yml

Spring Boot's `@ConfigurationProperties` binds your YAML definitions to `LlmOptions` beans. Define each LLM under the `embabel.ai.models` prefix, specifying unique names, providers, and credentials.

```yaml
embabel:
  ai:
    models:
      - name: openai-gpt4
        provider: openai
        apiKey: ${OPENAI_API_KEY}
        model: gpt-4
        temperature: 0.7
      - name: anthropic-claude
        provider: anthropic
        apiKey: ${ANTHROPIC_API_KEY}
        model: claude-2
        temperature: 0.6
    default: openai-gpt4
    roles:
      summarizer: anthropic-claude
      translator: openai-gpt4

```

Spring injects this list as distinct beans. The `default` key establishes the fallback when no specific criteria are provided, while the `roles` map enables semantic routing based on task type.

## Selecting LLMs at Runtime

The static helper methods in `LlmOptions` (lines 44-95) construct the appropriate `ModelSelectionCriteria` without manual bean wiring.

### By Explicit Name

Target a specific model configuration by its declared name:

```kotlin
import com.embabel.common.ai.model.LlmOptions

val claudeOpts = LlmOptions.withModel("anthropic-claude")

```

### By Role

Route requests based on functional roles defined in your YAML:

```kotlin
val summarizerOpts = LlmOptions.withLlmForRole("summarizer")

```

### Fallback Strategies

Define resilient routing that attempts multiple models in sequence:

```kotlin
val fallbackOpts = LlmOptions.withFirstAvailableLlmOf(
    "anthropic-claude", 
    "openai-gpt4"
)

```

Use the platform default when you want consistent behavior across the application:

```kotlin
val defaultOpts = LlmOptions.withDefaultLlm()

```

### Using Options with AiService

Pass the configured options to `AiService.chat()` to execute against the resolved LLM:

```kotlin
val response = aiService.chat(
    prompt = "Summarize the following article...",
    options = LlmOptions.withLlmForRole("summarizer")
)
println(response)

```

## Advanced Configuration and Provider Extensions

For provider-specific capabilities, use the `extensions` map in `LlmOptions`. Provider modules expose type-safe helpers to populate these values.

Enable Anthropic's caching extension (lines 92-104 in [`LlmOptions.kt`](https://github.com/embabel/embabel-agent/blob/main/LlmOptions.kt)):

```kotlin
val opts = LlmOptions.withModel("anthropic-claude")
    .withExtension("anthropicCaching", true)

```

Alternatively, use dedicated helpers if available:

```kotlin
val opts = LlmOptions.withModel("anthropic-claude")
    .withAnthropicCaching(true)

```

The `extensions` map allows the core framework to remain provider-agnostic while enabling access to unique features like thinking blocks, caching, or custom headers.

## Summary

- **Define LLMs** in [`application.yml`](https://github.com/embabel/embabel-agent/blob/main/application.yml) under `embabel.ai.models` with unique names and provider-specific credentials
- **Register providers** automatically via factories like [`OpenAiCompatibleModelFactory.kt`](https://github.com/embabel/embabel-agent/blob/main/OpenAiCompatibleModelFactory.kt) using Spring's auto-configuration
- **Select models** programmatically using `LlmOptions.withModel()`, `withLlmForRole()`, or `withFirstAvailableLlmOf()`
- **Resolve criteria** at runtime through the `criteria` getter in [`LlmOptions.kt`](https://github.com/embabel/embabel-agent/blob/main/LlmOptions.kt) which maps to concrete provider beans
- **Extend functionality** via the `extensions` map for provider-specific features without breaking abstraction

## Frequently Asked Questions

### How does Embabel discover available LLM providers?

Provider modules in the `embabel-agent` repository, such as `embabel-agent-openai`, register their beans through factory classes like [`OpenAiCompatibleModelFactory.kt`](https://github.com/embabel/embabel-agent/blob/main/OpenAiCompatibleModelFactory.kt). Spring Boot's component scan picks up these factories during auto-configuration, making the models available to the `ModelSelectionCriteria` resolution engine without explicit bean declarations.

### Can I use different API keys for different models?

Yes. Each entry under `embabel.ai.models` in your [`application.yml`](https://github.com/embabel/embabel-agent/blob/main/application.yml) defines its own `apiKey` field. You can configure distinct keys for OpenAI, Anthropic, Azure, or DashScope within the same application, allowing you to mix commercial and enterprise endpoints with separate authentication.

### What happens if the selected LLM is unavailable?

If you use `LlmOptions.withFirstAvailableLlmOf()`, Embabel attempts each model in the specified order until one succeeds. Without fallback configuration, the framework throws a resolution error if the criteria match an unavailable provider. Implementing fallback criteria is recommended for production resilience.

### How do I add custom provider-specific options?

Use the `withExtension()` method on `LlmOptions` to populate the `extensions` map with arbitrary key-value pairs. Provider modules read these values to enable features like Anthropic's caching or custom reasoning parameters. This mechanism keeps your code portable while accessing provider-specific capabilities.