How to Fine-Tune Embabel-Agent Models: A Complete Configuration Guide

Fine-tuning Embabel-Agent models requires exposing your custom model as a Spring AI ChatModel and configuring an Llm bean with specific options via LlmOptions to override default model IDs and generation parameters.

Embabel-Agent provides a flexible integration layer for Large Language Models (LLMs) through Spring AI, making it straightforward to incorporate fine-tuned models into your agent workflows. Because the framework delegates model management to Spring AI providers, you can use fine-tuned versions from OpenAI, Anthropic, or local deployments by updating configuration rather than modifying core logic. This guide demonstrates how to point your Embabel-Agent application to custom fine-tuned models using the official embabel/embabel-agent repository patterns.

Understanding the Architecture

Spring AI ChatModel Integration

Embabel-Agent relies on Spring AI's ChatModel interface to communicate with LLM providers. In embabel-agent-openai/src/main/kotlin/com/embabel/agent/openai/OpenAiConfiguration.kt, the framework defines beans that connect to provider APIs, using the model ID specified in your configuration.

When you fine-tune a model through a cloud provider, you receive a custom model identifier (e.g., ft:gpt-3.5-turbo-0613:my-org::MY_FINE_TUNED_MODEL). By substituting this ID for the default model name in your configuration, Spring AI routes requests to your fine-tuned endpoint automatically.

The LlmOptions Configuration Layer

The LlmOptions class, located in embabel-agent-common/embabel-agent-ai/src/main/kotlin/com/embabel/common/ai/model/LlmOptions.kt, provides a fluent builder for per-call hyperparameters. This includes temperature, topP, maxTokens, and crucially, the model identifier. When building an Llm bean, you can set default options that specify your fine-tuned model ID, ensuring all agent actions use the correct weights without manual intervention.

Step-by-Step Fine-Tuning Workflow

1. Prepare Your Fine-Tuned Model

First, create your fine-tuned model through your provider's standard workflow. For OpenAI, use the dashboard or CLI to initiate training, then note the resulting model ID. For local deployments using Ollama or LM Studio, ensure your fine-tuned weights are loaded and the service is accessible via base URL.

2. Add the Embabel-Agent Starter

Include the appropriate starter dependency in your pom.xml or build.gradle. For OpenAI models:

<dependency>
    <groupId>com.embabel.agent</groupId>
    <artifactId>embabel-agent-starter</artifactId>
    <version>0.3.0</version>
</dependency>

For local Ollama deployments, use the Ollama-specific starter instead:

<dependency>
    <groupId>com.embabel.agent</groupId>
    <artifactId>embabel-agent-starter-ollama</artifactId>
    <version>0.3.0</version>
</dependency>

3. Configure API Credentials

Set your provider API keys as environment variables or in application.properties:

OPENAI_API_KEY=sk-your-key-here
ANTHROPIC_API_KEY=sk-ant-your-key-here

4. Update Application Configuration

Modify src/main/resources/application.yml to reference your fine-tuned model ID instead of the base model:

spring:
  ai:
    openai:
      api-key: ${OPENAI_API_KEY}
      chat:
        model: ft:gpt-3.5-turbo-0613:my-org::MY_FINE_TUNED_MODEL
        temperature: 0.7
        max-tokens: 2000

This configuration directs the Spring AI ChatModel bean—defined in OpenAiConfiguration.kt—to use your custom model for all chat completions.

5. (Optional) Define Custom Llm Beans

For applications requiring multiple models (e.g., a base model for quick tasks and a fine-tuned model for specialized generation), declare explicit beans:

@Configuration
class CustomLlmConfig {

    @Bean
    fun fineTunedLlm(openAiChatModel: ChatModel): Llm {
        return Llm.builder()
            .chatModel(openAiChatModel)
            .defaultOptions(
                LlmOptions.withModel("ft:gpt-3.5-turbo-0613:my-org::MY_FINE_TUNED_MODEL")
                    .withTemperature(0.8)
                    .withMaxTokens(1500)
            )
            .build()
    }

    @Bean
    fun baseLlm(openAiChatModel: ChatModel): Llm {
        return Llm.builder()
            .chatModel(openAiChatModel)
            .defaultOptions(
                LlmOptions.withModel("gpt-3.5-turbo")
                    .withTemperature(0.2)
            )
            .build()
    }
}

6. Implement Agent Actions

Inject your custom Llm bean into agent classes and invoke it using ai.withLlm():

@Agent(description = "Generate specialized reports using fine-tuned knowledge")
class ReportAgent(
    private val fineTunedLlm: Llm
) {

    @Action
    fun createReport(input: UserInput, ai: Ai): Report {
        return ai.withLlm(fineTunedLlm)
            .createObject(
                "Generate a technical summary for: ${input.content}",
                Report::class.java
            )
    }
}

The ai.withLlm() method overrides the default LLM for that specific action, routing the prompt to your fine-tuned model with the parameters specified in your LlmOptions.

Configuration Examples

YAML-Based Configuration

For simple deployments where all agents should use the same fine-tuned model, YAML configuration suffices:

spring:
  ai:
    openai:
      api-key: ${OPENAI_API_KEY}
      chat:
        model: ft:gpt-4-0613:my-org::CUSTOM_MODEL_ID
        temperature: 0.6
        top-p: 0.9

This approach requires no additional code changes; the default Llm bean automatically inherits these settings.

Programmatic Bean Definition

When you need dynamic model selection or different parameters for different agent types, use explicit bean configuration:

@Configuration
class LlmBeans {

    @Bean
    fun cheapLlm(openAiChatModel: ChatModel): Llm = 
        Llm.builder()
            .chatModel(openAiChatModel)
            .defaultOptions(
                LlmOptions.withModel("gpt-3.5-turbo")
                    .withTemperature(0.2)
            )
            .build()

    @Bean
    fun premiumFineTunedLlm(openAiChatModel: ChatModel): Llm = 
        Llm.builder()
            .chatModel(openAiChatModel)
            .defaultOptions(
                LlmOptions.withModel("ft:gpt-4-0613:my-org::PREMIUM_TUNED")
                    .withTemperature(0.7)
                    .withMaxTokens(4096)
            )
            .build()
}

Multi-Model Agent Implementation

Create agents that intelligently select models based on context or budget:

@Agent(description = "Cost-aware recommendation engine")
class CostAwareAgent(
    private val cheapLlm: Llm,
    private val premiumFineTunedLlm: Llm
) {

    @Action
    fun recommend(request: RecommendationRequest, ai: Ai): Recommendation {
        val selectedLlm = if (request.budget < 0.05) cheapLlm else premiumFineTunedLlm
        
        return ai.withLlm(selectedLlm)
            .createObject(
                "Generate recommendation for ${request.topic}",
                Recommendation::class.java
            )
    }
}

Summary

• Embabel-Agent delegates model management to Spring AI, meaning you fine-tune through your provider (OpenAI, Anthropic, etc.) and reference the custom model ID in configuration.
• LlmOptions provides the fluent interface for setting model IDs, temperature, and token limits, located in embabel-agent-common/embabel-agent-ai/src/main/kotlin/com/embabel/common/ai/model/LlmOptions.kt.
• YAML configuration in application.yml is sufficient for single-model deployments using spring.ai.openai.chat.model.
• Explicit Llm beans allow you to maintain multiple model configurations simultaneously and inject them into specific agent actions via ai.withLlm().
• The OpenAiConfiguration.kt file in embabel-agent-openai demonstrates how the framework wires Spring AI beans, which automatically respect your fine-tuned model identifiers.

Frequently Asked Questions

How do I switch between a base model and a fine-tuned model in the same application?

Define separate Llm beans for each model variant, injecting different LlmOptions with distinct model IDs. Autowire both beans into your agent class, then use conditional logic within your @Action methods to select the appropriate model via ai.withLlm(baseLlm) or ai.withLlm(fineTunedLlm).

Can I use fine-tuned models from local providers like Ollama?

Yes. Add the embabel-agent-starter-ollama dependency and configure the base URL pointing to your local Ollama instance. Specify your fine-tuned model tag in LlmOptions.withModel("your-model-tag"), ensuring the model files are available locally.

Where do I set the temperature and max-tokens for a fine-tuned model?

Use the LlmOptions builder when constructing your Llm bean. Methods like withTemperature(), withMaxTokens(), and withModel() allow you to customize generation parameters. These settings apply to all calls made through that Llm instance, overriding any defaults from application.yml.

Do I need to modify the Embabel-Agent source code to use a custom fine-tuned model?

No. The framework is designed to accept any Spring AI-compatible ChatModel. You simply configure the model ID in your YAML or Java/Kotlin configuration classes. The OpenAiConfiguration.kt source demonstrates this extensibility by creating beans that respect external configuration properties.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}