How to Configure Embabel Agents to Use Multiple LLM Providers

Embabel agents can communicate with any LLM that Spring AI exposes a ChatModel for by adding the appropriate starter dependencies and optionally defining custom Llm beans, enabling runtime selection between providers like OpenAI, Anthropic, and Ollama.

The embabel/embabel-agent framework abstracts LLM interactions through Spring AI, allowing you to mix and match providers within a single application. By leveraging auto-configuration modules and the Llm abstraction, you can configure Embabel agents to use multiple LLM providers without refactoring your agent logic.

Add LLM Starter Dependencies

Embabel distributes LLM support through dedicated starter modules. Each starter pulls in the necessary Spring AI integration and registers a default Llm bean. To enable multiple providers, add each starter to your build file.

For Maven, include the artifacts in your pom.xml:

<dependency>
  <groupId>com.embabel.agent</groupId>
  <artifactId>embabel-agent-starter-openai</artifactId>
  <version>${embabel.agent.version}</version>
</dependency>

<dependency>
  <groupId>com.embabel.agent</groupId>
  <artifactId>embabel-agent-starter-anthropic</artifactId>
  <version>${embabel.agent.version}</version>
</dependency>

Each starter triggers a specific auto-configuration class:

• OpenAI: com.embabel.agent.autoconfigure.models.openai.AgentOpenAiAutoConfiguration located in embabel-agent-autoconfigure/models/embabel-agent-openai-autoconfigure/src/main/java/com/embabel/agent/autoconfigure/models/openai/AgentOpenAiAutoConfiguration.java
• Anthropic: com.embabel.agent.autoconfigure.models.anthropic.AgentAnthropicAutoConfiguration located in embabel-agent-autoconfigure/models/embabel-agent-anthropic-autoconfigure/src/main/java/com/embabel/agent/autoconfigure/models/anthropic/AgentAnthropicAutoConfiguration.java

Configure API Keys and Default Models

All providers follow the Spring AI property naming scheme. Store credentials in environment variables and reference them in application.yml to keep secrets out of source control:

spring:
  ai:
    openai:
      api-key: ${OPENAI_API_KEY}
      chat:
        model: gpt-4o
    anthropic:
      api-key: ${ANTHROPIC_API_KEY}
      chat:
        model: claude-3-sonnet-20240229

embabel:
  models:
    default-llm: openAiMini

The embabel.models.default-llm property determines which bean the framework injects when you call ai.withDefaultLlm().

Define Custom Llm Beans for Fine-Grained Control

While starters create default beans, defining your own Llm beans gives you precise control over model names, temperature, and other parameters. Use the LlmOptionsFactory provided by the Embabel AI module to build provider-specific instances:

@Configuration
public class MyLlmConfig {

    @Bean
    public Llm openAiMini(LlmOptionsFactory factory) {
        return factory.withModel(OpenAiModels.GPT_4_MINI)
                      .withTemperature(0.7)
                      .build();
    }

    @Bean
    public Llm anthropicClaude(LlmOptionsFactory factory) {
        return factory.withModel(AnthropicModels.CLAUDE_3_SONNET)
                      .withTemperature(0.6)
                      .build();
    }
}

This approach lets you expose multiple named beans for the same provider or tune hyperparameters per use case.

Select Providers at Runtime

With multiple Llm beans registered, switch between them programmatically using the Ai interface. Call withLlm() and pass either the bean name or the Llm instance directly:

// Use the OpenAI mini model by bean name
Writeup summary = ai.withLlm("openAiMini")
                    .createObject("Summarize this report", Writeup.class);

// Switch to Anthropic for a different task
Writeup analysis = ai.withLlm("anthropicClaude")
                     .createObject("Analyze sentiment", Writeup.class);

If you omit the provider and call ai.withDefaultLlm(), Embabel uses the bean specified in embabel.models.default-llm.

Mix Cloud and Local Providers

Embabel supports local inference through starters like Ollama. Add the dependency to run models locally alongside cloud providers:

<dependency>
  <groupId>com.embabel.agent</groupId>
  <artifactId>embabel-agent-starter-ollama</artifactId>
  <version>${embabel.agent.version}</version>
</dependency>

The AgentOllamaAutoConfiguration class in embabel-agent-autoconfigure/models/embabel-agent-ollama-autoconfigure/src/main/java/com/embabel/agent/autoconfigure/models/ollama/AgentOllamaAutoConfiguration.java wires an OllamaChatModel bean. You can then combine OpenAI for high-quality generation and Ollama for fast, cost-free local tasks within the same agent.

Complete Example: Kotlin Agent with Multiple Providers

@Agent(description = "Demo agent that uses two LLMs")
class MultiLlmAgent(private val ai: Ai) {

    @Action
    fun summarizeWithOpenAi(text: String): String =
        ai.withLlm("openAiMini")
          .createObject("Summarize: $text", String::class.java)

    @Action
    fun translateWithAnthropic(text: String): String =
        ai.withLlm("anthropicClaude")
          .createObject("Translate to French: $text", String::class.java)
}

Summary

• Add starters for each LLM provider you need; each starter registers a default Llm bean via auto-configuration classes like AgentOpenAiAutoConfiguration.
• Configure credentials using standard Spring AI properties in application.yml and select a default provider with embabel.models.default-llm.
• Define custom beans using LlmOptionsFactory to expose multiple models from the same provider with different temperatures or model names.
• Switch at runtime by calling ai.withLlm(beanName) or ai.withLlm(Llm) to route specific actions to specific providers.
• Combine cloud and local models by including starters like embabel-agent-starter-ollama alongside cloud providers.

Frequently Asked Questions

Can I use more than two LLM providers in the same Embabel application?

Yes. There is no hard limit on the number of providers. Add the respective starter dependency for each provider—OpenAI, Anthropic, Ollama, or OCI GenAI—and define corresponding Llm beans. Each bean operates independently within the same Spring context.

How do I change the default LLM without modifying code?

Update the embabel.models.default-llm property in your application.yml or set it via an environment variable. When you call ai.withDefaultLlm(), Embabel resolves the bean name from this property, allowing you to switch defaults across environments without touching Java or Kotlin source files.

Do I need to define custom Llm beans to use multiple providers?

No. The starters automatically create default beans for each provider. However, custom beans are required if you need non-default model names, specific temperature settings, or cost-tracking wrappers. Use LlmOptionsFactory to construct these instances consistently.

Which local LLM providers does Embabel support?

Embabel supports any local provider that Spring AI integrates with, including Ollama. The embabel-agent-starter-ollama dependency triggers AgentOllamaAutoConfiguration, which wires a local OllamaChatModel that behaves identically to cloud-based Llm beans in your agent code.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}