How to Configure Multiple LLM Providers (OpenAI, Anthropic, and Ollama) in Embabel

Embabel allows you to configure multiple LLM providers simultaneously by adding provider-specific Spring Boot starter dependencies and defining credentials under the embabel.agent.platform.models.<provider> namespace in your application configuration.

Embabel (from the embabel/embabel-agent repository) is a Spring AI-based framework that enables seamless integration of multiple large language model providers within a single application. Unlike single-provider setups, Embabel's architecture supports concurrent initialization of OpenAI, Anthropic, and Ollama through provider-specific autoconfiguration modules and a unified property-based configuration model.

Understanding Embabel's Multi-Provider Architecture

Embabel's core architecture decouples LLM provider integration into provider-specific starter modules that autoconfigure distinct LlmService beans. Each starter—such as embabel-agent-starter-openai or embabel-agent-starter-anthropic—registers its own chat model, embedding model, and REST client based on the unified configuration class LlmOptionsProperties.kt.

According to the Embabel source code, the configuration hierarchy starts with the base prefix embabel.agent.platform.models defined in LlmOptionsProperties.kt (embabel-agent-common/embabel-agent-ai/src/main/kotlin/com/embabel/common/ai/model/LlmOptionsProperties.kt). Provider-specific modules extend this base to support unique parameters like Anthropic's caching headers or Ollama's local endpoint requirements.

Step 1: Add Provider-Specific Starter Dependencies

To enable multiple providers, include the corresponding starter artifacts in your build file. You need the OpenAI starter for OpenAI (and Ollama via compatibility), plus the dedicated Anthropic starter.

// build.gradle.kts
implementation("com.embabel.agent:emabel-agent-starter-openai:${embabelAgentVersion}")
implementation("com.embabel.agent:emabel-agent-starter-anthropic:${embabelAgentVersion}")
// Ollama uses the OpenAI-compatible starter (no separate Ollama starter required)
implementation("com.embabel.agent:emabel-agent-starter-openai:${emabelAgentVersion}")

If using Maven, add equivalent <dependency> blocks with the same groupId and artifactId values.

Step 2: Configure Provider Credentials and Endpoints

Define each provider's credentials and model lists under the embabel.agent.platform.models hierarchy in application.yml. The OpenAI configuration uses the openai key, Anthropic uses anthropic, and Ollama uses the openai-custom block because it implements the OpenAI-compatible REST schema.

# OpenAI (default provider)

embabel:
  agent:
    platform:
      models:
        openai:
          api-key: ${OPENAI_API_KEY}
          model-list:
            - gpt-4o
            - gpt-4-turbo

# Anthropic

embabel:
  agent:
    platform:
      models:
        anthropic:
          api-key: ${ANTHROPIC_API_KEY}
          model-list:
            - claude-3-5-sonnet-20240620
            - claude-3-opus-20240229
          max-attempts: 3
          backoff-millis: 500
          backoff-multiplier: 2.0

# Ollama (OpenAI-compatible endpoint)

embabel:
  agent:
    platform:
      models:
        openai-custom:
          api-key: ""
          base-url: http://localhost:11434/v1
          model-list:
            - llama2
            - mistral

The AnthropicModelsConfig.kt file (embabel-agent-anthropic-autoconfigure/src/main/kotlin/com/embabel/agent/config/models/anthropic/AnthropicModelsConfig.kt) handles provider-specific extensions like backoff configuration, while OpenAiCustomModelsConfig.kt (embabel-agent-autoconfigure/models/embabel-agent-openai-custom-autoconfigure/src/main/kotlin/com/embabel/agent/config/models/openai/custom/OpenAiCustomModelsConfig.kt) enables the generic OpenAI-compatible layer used for Ollama.

Step 3: Inject and Use Multiple LlmService Beans

Once configured, Embabel registers separate LlmService beans for each provider. Inject them into your components to route requests to specific backends.

import com.embabel.agent.ai.LlmService
import org.springframework.beans.factory.annotation.Autowired
import org.springframework.stereotype.Component

@Component
class MultiProviderChat @Autowired constructor(
    private val openAiChat: LlmService,
    private val anthropicChat: LlmService,
    private val ollamaChat: LlmService
) {
    fun routeToProvider(provider: String, prompt: String): String {
        return when (provider) {
            "openai" -> openAiChat.chat(prompt).content
            "anthropic" -> anthropicChat.chat(prompt).content
            "ollama" -> ollamaChat.chat(prompt).content
            else -> throw IllegalArgumentException("Unknown provider")
        }
    }
}

Each injected LlmService instance is automatically bound to its respective provider configuration defined in Step 2.

Step 4: Verify Provider Initialization at Runtime

Embabel logs a concise initialization summary at startup, confirming which providers loaded successfully. Look for output similar to:

OpenAI: Initialized 1 LLM(s) and 1 embedding(s)
Anthropic: Initialized 1 LLM(s) and 0 embedding(s)
OpenAI-Custom: Initialized 1 LLM(s) and 0 embedding(s)

If a provider fails to initialize (e.g., missing API key or unreachable endpoint), the log line will include the specific failure reason without preventing other providers from starting.

Customizing Provider-Specific Extensions

Beyond basic credential configuration, the Anthropic module exposes specialized utilities for cache token tracking via extension functions like anthropicCacheCreationTokens(usage). These provider-specific helpers are located in the respective autoconfigure modules and become available when you import the corresponding starter.

For Ollama deployments, extend the openai-custom block in OpenAiCustomModelsConfig.kt to adjust base-url values for self-hosted endpoints or Azure OpenAI compatibility.

Key Implementation Files

• LlmOptionsProperties.kt (embabel-agent-common/embabel-agent-ai/src/main/kotlin/com/embabel/common/ai/model/LlmOptionsProperties.kt): Defines the base configuration properties shared across all providers.
• OpenAiModelsConfig.kt (embabel-agent-openai-autoconfigure/src/main/kotlin/com/embel/agent/config/models/openai/OpenAiModelsConfig.kt): Configures the default OpenAI provider including base URL and API key handling.
• AnthropicModelsConfig.kt (embabel-agent-anthropic-autoconfigure/src/main/kotlin/com/embel/agent/config/models/anthropic/AnthropicModelsConfig.kt): Handles Anthropic-specific settings like model lists and retry backoff strategies.
• OpenAiCustomModelsConfig.kt (embabel-agent-autoconfigure/models/embabel-agent-openai-custom-autoconfigure/src/main/kotlin/com/embel/agent/config/models/openai/custom/OpenAiCustomModelsConfig.kt): Enables OpenAI-compatible providers like Ollama through generic configuration.
• ProviderDetection.kt (embabel-agent-common/embel-agent-byok/src/main/kotlin/com/embel/common/byok/ProviderDetection.kt): Supports bring-your-own-key (BYOK) detection logic for dynamic provider identification.
• application-test.yml (embel-agent-api/src/test/resources/application-test.yml): Reference test configuration demonstrating multi-provider YAML structure.

Summary

Configuring multiple LLM providers in Embabel requires three steps:

• Add starter dependencies for each provider (embel-agent-starter-openai, embel-agent-starter-anthropic).
• Define credentials under embabel.agent.platform.models.<provider> using environment variables or hardcoded values in application.yml.
• Inject LlmService beans into your Spring components—one per configured provider.

Because Ollama implements the OpenAI-compatible REST schema, it integrates through the generic openai-custom configuration block without requiring a dedicated starter, simplifying local development alongside cloud providers.

Frequently Asked Questions

Can I use Ollama without a dedicated Embabel starter?

Yes. Ollama implements an OpenAI-compatible REST API, so you configure it using the embel-agent-starter-openai dependency and the openai-custom configuration block with base-url: http://localhost:11434/v1. This architecture avoids the need for provider-specific code while maintaining full compatibility with Ollama's local model serving.

How do I secure API keys when configuring multiple providers?

Embabel supports Spring Boot's property placeholder syntax. Instead of hardcoding keys, use environment variable references like api-key: ${OPENAI_API_KEY} or api-key: ${ANTHROPIC_API_KEY} in your application.yml. The ProviderDetection.kt utility also supports bring-your-own-key (BYOK) patterns for runtime key injection.

What happens if one provider fails to initialize?

Embabel initializes providers independently. If one provider (e.g., Anthropic) fails due to a missing API key or network timeout, the application context still loads successfully for other configured providers. Check the startup logs for specific initialization failure messages under the provider name (e.g., "Anthropic: Failed to initialize").

Can I configure multiple instances of the same provider?

While the analysis demonstrates single instances per provider type, the openai-custom block can be used to configure additional OpenAI-compatible endpoints (such as Azure OpenAI or separate Ollama instances) by specifying different base-url values and distinct bean names, though this may require additional @Qualifier annotations in your injection points depending on your specific Spring configuration.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}