How to Configure Different LLM Models for Each Agent in MathModelAgent Using Environment Variables

MathModelAgent uses Pydantic Settings to map environment variables like COORDINATOR_MODEL and MODELER_MODEL to specific agents, allowing each to use different LLM providers without code changes.

Configuring different LLM models for each agent in the jihe520/mathmodelagent repository is straightforward using environment variables. The system employs Pydantic Settings to load configuration from .env files, enabling independent control over the Coordinator, Modeler, Coder, and Writer agents. This approach lets you mix providers—such as OpenAI for coordination and DeepSeek for coding—simply by updating environment variables.

Understanding the Configuration Architecture

The configuration system is defined in backend/app/config/setting.py using Pydantic's BaseSettings class. This automatically reads environment variables and maps them to Python attributes:

# backend/app/config/setting.py

class Settings(BaseSettings):
    COORDINATOR_MODEL: Optional[str] = None
    COORDINATOR_API_KEY: Optional[str] = None
    COORDINATOR_BASE_URL: Optional[str] = None
    
    MODELER_MODEL: Optional[str] = None
    MODELER_API_KEY: Optional[str] = None
    MODELER_BASE_URL: Optional[str] = None
    
    CODER_MODEL: Optional[str] = None
    CODER_API_KEY: Optional[str] = None
    CODER_BASE_URL: Optional[str] = None
    
    WRITER_MODEL: Optional[str] = None
    WRITER_API_KEY: Optional[str] = None
    WRITER_BASE_URL: Optional[str] = None
    
    model_config = SettingsConfigDict(
        env_file=".env.dev",
        env_file_encoding="utf-8",
        extra="allow",
    )

When the module initializes, settings = Settings() reads values from the environment file or process environment. Each agent requires three variables: MODEL (the identifier), API_KEY (authentication), and BASE_URL (the endpoint).

Step-by-Step Configuration Guide

Create Your Environment File

Create a .env.dev file in the repository root. Assign distinct models to each agent based on their specialized tasks:

# .env.dev - Repository root

ENV=dev

# Coordinator - Lightweight orchestration

COORDINATOR_MODEL=gpt-4o-mini
COORDINATOR_API_KEY=sk-coordinator-key
COORDINATOR_BASE_URL=https://api.openai.com/v1

# Modeler - Mathematical reasoning

MODELER_MODEL=claude-3-5-sonnet-20240620
MODELER_API_KEY=sk-modeler-key
MODELER_BASE_URL=https://api.anthropic.com/v1

# Coder - Code generation

CODER_MODEL=deepseek-coder-v2
CODER_API_KEY=sk-coder-key
CODER_BASE_URL=https://api.deepseek.com/v1

# Writer - Document generation

WRITER_MODEL=gemini-1.5-flash
WRITER_API_KEY=sk-writer-key
WRITER_BASE_URL=https://generativelanguage.googleapis.com/v1

Verify Settings Loading

Import the settings object anywhere in your application to verify configuration:

from app.config.setting import settings

print(f"Coordinator: {settings.COORDINATOR_MODEL}")
print(f"Modeler: {settings.MODELER_MODEL}")
print(f"Coder: {settings.CODER_MODEL}")
print(f"Writer: {settings.WRITER_MODEL}")

How Environment Variables Flow to Agents

The LLM Factory (backend/app/core/llm/llm_factory.py) bridges configuration and agent initialization. It constructs LLM instances using the environment values:

# backend/app/core/llm/llm_factory.py

coordinator_llm = LLM(
    api_key=settings.COORDINATOR_API_KEY,
    model=settings.COORDINATOR_MODEL,
    base_url=settings.COORDINATOR_BASE_URL,
    task_id=self.task_id,
)

modeler_llm = LLM(
    api_key=settings.MODELER_API_KEY,
    model=settings.MODELER_MODEL,
    base_url=settings.MODELER_BASE_URL,
    task_id=self.task_id,
)

coder_llm = LLM(
    api_key=settings.CODER_API_KEY,
    model=settings.CODER_MODEL,
    base_url=settings.CODER_BASE_URL,
    task_id=self.task_id,
)

writer_llm = LLM(
    api_key=settings.WRITER_API_KEY,
    model=settings.WRITER_MODEL,
    base_url=settings.WRITER_BASE_URL,
    task_id=self.task_id,
)

These instances are injected into the workflow in backend/app/core/workflow.py, ensuring each agent uses its designated model:

from app.core.workflow import MathModelWorkFlow

workflow = MathModelWorkFlow(
    task_id=task_id,
    coordinator_llm=coordinator_llm,
    modeler_llm=modeler_llm,
    coder_llm=coder_llm,
    writer_llm=writer_llm,
)

Runtime Configuration Updates

You can modify models without restarting the server using the /save-api-config endpoint defined in backend/app/routers/modeling_router.py. This endpoint mutates the global settings object directly:

# backend/app/routers/modeling_router.py

if request.modeler:
    settings.MODELER_API_KEY = request.modeler.get("apiKey", "")
    settings.MODELER_MODEL = request.modeler.get("modelId", "")
    settings.MODELER_BASE_URL = request.modeler.get("baseUrl", "")

Update models dynamically via HTTP request:

curl -X POST http://localhost:8000/api/save-api-config \
  -H "Content-Type: application/json" \
  -d '{
    "modeler": {
      "modelId": "gpt-4o",
      "apiKey": "sk-new-key",
      "baseUrl": "https://api.openai.com/v1"
    },
    "coder": {
      "modelId": "deepseek-coder-v2",
      "apiKey": "sk-new-key",
      "baseUrl": "https://api.deepseek.com/v1"
    }
  }'

Complete Configuration Example

Here is a production-ready configuration mixing multiple providers:

# .env.production

ENV=production

COORDINATOR_MODEL=gpt-4o
COORDINATOR_API_KEY=${OPENAI_API_KEY}
COORDINATOR_BASE_URL=https://api.openai.com/v1

MODELER_MODEL=claude-3-opus-20240229
MODELER_API_KEY=${ANTHROPIC_API_KEY}
MODELER_BASE_URL=https://api.anthropic.com/v1

CODER_MODEL=deepseek-coder-v2
CODER_API_KEY=${DEEPSEEK_API_KEY}
CODER_BASE_URL=https://api.deepseek.com/v1

WRITER_MODEL=gpt-4o-mini
WRITER_API_KEY=${OPENAI_API_KEY}
WRITER_BASE_URL=https://api.openai.com/v1

Initialize the complete workflow with these settings:

from app.core.llm.llm_factory import LLMFactory
from app.core.workflow import MathModelWorkFlow

task_id = "task-001"
factory = LLMFactory(task_id)

coord_llm, modeler_llm, coder_llm, writer_llm = factory.get_all_llms()

workflow = MathModelWorkFlow(
    task_id=task_id,
    coordinator_llm=coord_llm,
    modeler_llm=modeler_llm,
    coder_llm=coder_llm,
    writer_llm=writer_llm,
)

result = await workflow.run(problem_description="Optimize supply chain logistics")

Summary

• Environment-based configuration: Define COORDINATOR_MODEL, MODELER_MODEL, CODER_MODEL, and WRITER_MODEL in .env.dev or .env.production.
• Pydantic Settings: The Settings class in backend/app/config/setting.py automatically loads and validates these variables.
• Per-agent LLM instantiation: LLMFactory in backend/app/core/llm/llm_factory.py creates separate LLM instances for each agent using the environment values.
• Runtime flexibility: Use the /save-api-config endpoint to update models dynamically without redeploying.

Frequently Asked Questions

Can I use the same LLM provider for all agents?

Yes. Set identical BASE_URL values and corresponding API_KEY variables for all four agents. However, using different models (e.g., gpt-4o for Modeler and gpt-4o-mini for Coordinator) optimizes cost and performance based on each agent's specific task requirements.

Do I need to restart the server after changing environment variables?

Only if you modify the .env file directly. Changes made through the /save-api-config API endpoint take effect immediately for subsequent tasks because the endpoint updates the global settings object in memory. File-based changes require a restart to reload the Settings class.

What happens if I leave an environment variable empty?

The Pydantic model defines these fields as Optional[str] = None. If omitted, the value defaults to None, which will likely cause authentication or connection errors when the LLMFactory attempts to initialize that specific agent. Each agent requires valid MODEL, API_KEY, and BASE_URL values to function.

Where should I place the .env.dev file?

Place it in the repository root directory (same level as the backend folder). The SettingsConfigDict in backend/app/config/setting.py specifies env_file=".env.dev" as the default location. You can override this by setting the ENV environment variable or modifying the env_file path in the configuration.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}