How to Configure BYOK with ProviderDetection for LLM APIs in Embabel

Embabel automatically detects and configures LLM providers by scanning for non-empty API keys in your application configuration, enabling zero-code provider switching through the ProviderResolver mechanism.

The embabel/embabel-agent repository implements a Bring Your Own Key (BYOK) strategy that eliminates hard-coded provider selection. By leveraging ProviderDetection, the framework automatically registers the appropriate LLM client based solely on the presence of valid credentials in your Spring configuration, routing requests to providers like OpenAI, Azure OpenAI, or Mistral without manual bean wiring.

How Provider Detection Works in Embabel

The detection mechanism relies on Spring Boot auto-configuration classes that conditionally create beans only when an api-key property is present. The core selection logic resides in ProviderResolver.kt, which scans all available ProviderInitialization beans and selects the first with a non-empty credential.

The ProviderResolver Selection Logic

In embabel-agent-api/src/main/kotlin/com/embabel/agent/spi/ProviderResolver.kt, the resolution logic prioritizes the first provider with a valid BYOK:

fun resolve(): ProviderInitialization {
    // `providers` is a List<ProviderInitialization> supplied by Spring
    val byokProvider = providers.firstOrNull { it.apiKey?.isNotBlank() == true }
    return byokProvider ?: defaultProvider
}

This method is invoked by the AgentEngine during invocation creation, ensuring dynamic provider selection at runtime without hard-coded routing.

Configuration Properties Architecture

Each provider module exposes a properties class that binds to your application.yml. For example, OpenAiProviderProperties.java in embabel-agent-autoconfigure/models/embabel-agent-openai-autoconfigure/src/main/java/com/embabel/agent/config/models/openai/ maps the embabel.agent.providers.openai namespace to typed fields. The corresponding OpenAiAutoConfiguration.java creates the OpenAiClient bean only when apiKey is non-null, preventing unnecessary bean instantiation for unused providers.

Step-by-Step BYOK Configuration

Follow these steps to configure Bring Your Own Key with automatic provider detection.

1. Add the Provider Module

Include the provider-specific auto-configuration dependency in your build file:

// build.gradle.kts
implementation("com.embabel:embabel-agent-openai-autoconfigure")

For Maven, add the equivalent dependency to your pom.xml.

2. Supply Your API Key

Configure your personal API key in application.yml or through environment variables:

embabel:
  agent:
    providers:
      openai:
        api-key: ${OPENAI_API_KEY}   # Your personal key

        model: gpt-4o                # Optional default model

Other providers follow the same pattern under embabel.agent.providers.azure-openai.api-key or embabel.agent.providers.mistralai.api-key.

3. Optional: Force a Default Provider

When multiple keys are present, explicitly set the preferred provider to override the first-match behavior:

embabel:
  agent:
    provider:
      default: openai   # Must match the provider's config key

4. Verify Detection

Create a test to validate that Embabel correctly identifies your BYOK provider:

@Autowired lateinit var providerResolver: ProviderResolver

@Test 
fun `detects BYOK provider`() {
    val init = providerResolver.resolve()
    assertEquals("openai", init.providerName)
}

The ProviderResolverTest class in embabel-agent-api/src/test/kotlin/ demonstrates this validation pattern used in production.

Accessing the Resolved Provider in Application Code

Once configured, inject the ProviderResolver to access the detected provider dynamically:

@Service
class ChatService(@Autowired private val resolver: ProviderResolver) {

    fun ask(prompt: String): String {
        val provider = resolver.resolve()          // BYOK detection
        val client = provider.createClient()       // Provider-specific client
        return client.generateText(prompt)         // Unified API across providers
    }
}

Each ProviderInitialization implementation provides a createClient() method returning a concrete client (e.g., OpenAiClient) while exposing the provider name and available models through the initialization metadata.

Observability and Tracing

Embabel exposes the selected provider through OpenTelemetry-compatible tracing. The ChatModelObservationFilter.java in embabel-agent-observability/src/main/java/com/embabel/agent/observability/tracing/ automatically adds gen_ai.provider.name to every tracing span, allowing you to monitor which LLM provider handles each request in your observability platform.

Summary

• BYOK configuration requires only setting the api-key property under embabel.agent.providers.{provider-name} in your Spring configuration.
• ProviderDetection automatically selects the first provider with a non-blank API key via ProviderResolver.kt, or respects the explicit embabel.agent.provider.default setting if configured.
• Conditional bean creation in auto-configuration classes (such as OpenAiAutoConfiguration.java) ensures only the detected provider's client is instantiated.
• Observability integration exposes the provider name as gen_ai.provider.name in distributed traces through ChatModelObservationFilter.java for operational visibility.

Frequently Asked Questions

Can I configure multiple LLM providers simultaneously?

Yes. You can supply API keys for multiple providers in your configuration. Embabel will select the first provider with a non-empty key, or respect your embabel.agent.provider.default setting if specified. Each provider module creates its beans conditionally, so unused providers consume no runtime resources.

How does Embabel handle missing or invalid API keys?

If no provider has a configured API key, the ProviderResolver returns the defaultProvider, which may be null depending on your configuration. The specific auto-configuration classes use @ConditionalOnProperty annotations to prevent bean creation when api-key is absent, avoiding application startup failures while allowing graceful degradation.

Is the API key ever logged or exposed in traces?

No. The api-key property is marked as sensitive in the properties classes and is never emitted in logs or tracing spans. Only the provider name (e.g., "openai") is exposed through the gen_ai.provider.name attribute in traces via ChatModelObservationFilter.java, never the credential itself.

Can I switch providers at runtime without restarting?

The ProviderResolver resolves the provider during the invocation creation phase, meaning it checks the current state of ProviderInitialization beans. However, standard Spring configuration requires restart to change property values. For true runtime switching, you would need to implement custom logic to refresh the ProviderInitialization beans or use Spring Cloud Config for dynamic property updates.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}