How to Configure Multiple Agents with Different LLM Providers in DAT

Configure multiple agents with different LLM providers by defining each model in the global llms array, then referencing specific model names in each agent's configuration section via the default-llm key.

The DAT framework (junjiem/dat) enables you to run multiple Askdata agents within a single project, each powered by distinct large language models. This architecture allows you to optimize costs and performance by assigning lightweight models to simple tasks and powerful models to complex reasoning workflows, all controlled through the dat_project.yaml configuration file.

Understanding the Architecture

The multi-agent, multi-LLM system in DAT follows a factory-based instantiation pattern. When you define agents in your project YAML, ProjectUtil.createAskdataAgent() handles the orchestration by loading the project configuration, building a ContentStore, and delegating to the appropriate AskdataAgentFactory based on the agent's provider field.

For agentic implementations, AgenticAskdataAgentFactory parses the agent-specific configuration options—specifically default-llm and optionally sql-generation-llm—and selects the corresponding ChatModelInstance from the globally defined LLM list. The resulting AskdataAgent uses the selected model for all conversations and tool invocations.

Step-by-Step Configuration Guide

1. Define LLM Providers in the Global Configuration

First, declare all available models in the top-level llms array of your dat_project.yaml. Each entry requires a unique name that agents will reference later.

llms:
  - name: openai-gpt4
    provider: openai
    configuration:
      api-key: ${OPENAI_API_KEY}
      model: gpt-4
  - name: anthropic-claude
    provider: anthropic
    configuration:
      api-key: ${ANTHROPIC_API_KEY}
      model: claude-2
  - name: azure-gpt35
    provider: azure-openai
    configuration:
      api-key: ${AZURE_API_KEY}
      endpoint: ${AZURE_ENDPOINT}
      model: gpt-35-turbo

2. Configure Individual Agents with Specific LLMs

Next, define your agents in the agents array. Use the configuration section to bind each agent to a specific LLM via the default-llm key. You can also specify a separate sql-generation-llm for text-to-SQL tasks.

agents:
  # Agent using OpenAI GPT-4 for sales queries

  - name: sales-assistant
    description: "Answers sales-related queries with high accuracy"
    provider: agentic
    configuration:
      default-llm: openai-gpt4
      sql-generation-llm: openai-gpt4
      max-messages: 150
      data-preview: true

  # Agent using Anthropic Claude for research tasks

  - name: research-assistant
    description: "Handles research-oriented questions"
    provider: agentic
    configuration:
      default-llm: anthropic-claude
      max-tools-invocations: 5
      human-in-the-loop: false

  # Agent using Azure OpenAI for compliance-sensitive operations

  - name: compliance-auditor
    description: "Audits compliance queries"
    provider: agentic
    configuration:
      default-llm: azure-gpt35
      sql-generation-llm: openai-gpt4

3. Optional: Restrict Semantic Models per Agent

You can further specialize agents by limiting which semantic models they access. This reduces retrieval latency and prevents context contamination between domains.

  - name: medical-assistant
    provider: agentic
    configuration:
      default-llm: openai-gpt4
    semantic_models:
      - medical_records
    semantic_model_tags:
      - health

Internal Implementation Details

The configuration parsing happens through several coordinated components in the DAT source code.

Project Loading: ProjectUtil.loadProject() in dat-sdk/src/main/java/ai/dat/boot/utils/ProjectUtil.java parses the YAML into a DatProject instance, extracting both the global llms list and the agents configurations.

Factory Resolution: When ProjectUtil.createAskdataAgent() is invoked, it retrieves the appropriate factory via AskdataAgentFactoryManager.getFactory(agentConfig.getProvider()) (defined in dat-core/src/main/java/ai/dat/core/factories/AskdataAgentFactoryManager.java).

Configuration Binding: The AgenticAskdataAgentFactory (in dat-agents/dat-agent-agentic/src/main/java/ai/dat/agent/agentic/AgenticAskdataAgentFactory.java) reads the ReadableConfig object and extracts:

• default-llm: Maps to a ChatModelInstance from the global LLM list
• sql-generation-llm: Optional separate model for SQL generation
• Additional options like max-messages, human-in-the-loop, etc.

Validation: FactoryUtil.validateFactoryOptions ensures that the LLM names referenced in agent configurations actually exist in the global llms array, preventing runtime errors.

Practical Java Example

To instantiate a specific agent programmatically:

import ai.dat.boot.utils.ProjectUtil;
import ai.dat.core.agent.AskdataAgent;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.Collections;
import java.util.Map;

public class MultiAgentExample {
    public static void main(String[] args) {
        Path projectPath = Paths.get("/path/to/your/dat_project.yaml");
        String agentName = "sales-assistant";  // Must match YAML name
        Map<String, Object> variables = Collections.emptyMap();

        // Creates the agent with its configured LLM (OpenAI GPT-4)
        AskdataAgent agent = ProjectUtil.createAskdataAgent(
            projectPath, 
            agentName, 
            variables
        );

        // Execute a query
        String question = "What was the total revenue last quarter?";
        var response = agent.ask(question, Collections.emptyList());
        
        // Process the stream
        response.events().forEach(event -> 
            System.out.println(event.content())
        );
    }
}

The same code works for any agent defined in your YAML; the specific LLM provider and model are automatically injected based on the agent's default-llm configuration.

Key Source Files Reference

File	Role
dat-sdk/src/main/java/ai/dat/core/data/project/AgentConfig.java	POJO representing a single agent entry (name, provider, semantic_models, configuration).
dat-agents/dat-agent-agentic/src/main/java/ai/dat/agent/agentic/AgenticAskdataAgentFactory.java	Parses agent-specific configuration (including default-llm) and builds the concrete AskdataAgent.
dat-sdk/src/main/java/ai/dat/boot/utils/ProjectUtil.java	Loads the project YAML, creates the ContentStore, selects the proper factory, and returns a ready-to-use agent.
dat-sdk/src/main/resources/templates/project_yaml_template.jinja	Template used by the CLI to generate a starter dat_project.yaml; shows where to place LLM and agent sections.
dat-core/src/main/java/ai/dat/core/factories/AskdataAgentFactoryManager.java	Manages factory registration and resolution based on the provider string.
dat-servers/dat-server-openapi/src/main/java/ai/dat/server/openapi/controller/InfoController.java	Exposes /agents endpoint to verify that multiple agents are correctly registered.

Summary

• Global LLM Registry: Define all available models in the top-level llms array with unique names.
• Agent-Specific Binding: Use the default-llm key in each agent's configuration section to select which model powers that agent.
• Factory Pattern: The agentic provider uses AgenticAskdataAgentFactory to parse configurations and instantiate agents with the correct ChatModelInstance.
• Validation: The framework validates that referenced LLM names exist in the global registry before runtime.
• Flexibility: You can mix providers (OpenAI, Anthropic, Azure) within one project and even assign different models for general chat versus SQL generation via sql-generation-llm.

Frequently Asked Questions

Can I use the same LLM for multiple agents?

Yes. Multiple agents can reference the same LLM name in their default-llm configuration. The ChatModelInstance is shared according to the factory's implementation, but each agent maintains its own conversation state and tool configuration.

What happens if I specify an LLM name that doesn't exist in the global list?

The framework performs validation during agent creation. FactoryUtil.validateFactoryOptions checks that the default-llm and sql-generation-llm values match names defined in the top-level llms array. If a name is not found, agent instantiation fails with a configuration error before the agent can process any requests.

Can I assign different LLMs for chat and SQL generation within the same agent?

Yes. In addition to default-llm, you can specify sql-generation-llm in the agent's configuration section. This allows you to use a lightweight model for general conversation while employing a more capable model (such as GPT-4) specifically for generating complex SQL queries, optimizing both cost and performance.

How do I verify that all my agents are correctly registered with their LLMs?

You can query the /agents endpoint exposed by InfoController.java in the OpenAPI server module. This returns a list of all registered agents in the project, allowing you to confirm that each agent is present and that its configuration (including the linked LLM) has been loaded correctly without parsing errors.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}