How the OpenMAIC Orchestration System Uses LLMs for Agent Decision-Making

Yes, the OpenMAIC orchestration system uses Large Language Models (LLMs) as the primary decision-making engine, invoking the callLLM function from @/lib/ai/llm to generate agent actions, skills, and voice designs based on session context.

The THU-MAIC/OpenMAIC repository implements an agentic framework where LLMs serve as the policy layer rather than hard-coded rules. When an agent needs to determine its next action—whether filling a thinking gap, generating a voice design, or resolving a classroom prompt—the orchestration layer calls the centralized LLM wrapper to synthesize decisions from contextual data stored in the session store.

The LLM-Centric Architecture

The Core Wrapper Module (@/lib/ai/llm)

At the heart of OpenMAIC's decision-making lies the callLLM function exported from @/lib/ai/llm. This module abstracts provider-specific implementations, allowing the orchestration layer to transmit prompts containing session state, agent roles, and user context to external providers like OpenAI or Anthropic.

API Route Integration

Server-side decision entry points reside in /api/agent/* routes (e.g., agent-skills endpoints). When an agent requires a new decision, these routes construct prompts and invoke callLLM with usage source identifiers such as 'agentDecision', bridging HTTP requests with LLM inference.

The 5-Step Decision Pipeline

OpenMAIC processes agent decisions through a structured pipeline that transforms LLM outputs into UI updates:

1. Request Entry: API routes under /api/agent/* receive requests when an agent needs a new decision.

2. LLM Invocation: The @/lib/ai/llm module's callLLM function transmits the constructed prompt to configured providers.

3. Response Handling: Workbench helpers like skillDisplayLabel and skillTitle, along with parseSkillFromText, extract structured data (skill handles, voice specifications) from the LLM's text output and persist them to the session store.

4. UI Synchronization: React hooks useAgentSkills and useWorkspacePaneNavigation trigger re-renders, displaying the generated decision in timelines or classroom views while handling loading states.

5. Fallback Handling: If callLLM fails, AgentSkillsError and invalidateAgentSkills logic serve cached skill lists or error UIs, ensuring the orchestration remains robust.

The "LLM-Gap" Pattern and Thinking States

The LLM serves as the decisive engine for any "thinking" step. When the UI renders an "LLM-gap" row—as verified in tests/workbench/agent-end-gap.test.ts—this indicates the client has dispatched a request to callLLM and is awaiting the model's output to fill the timeline gap. This design makes the LLM the policy that determines the next agent action.

Implementation Examples

The following code demonstrates the integration between React hooks, the LLM wrapper, and server-side orchestration:

// React hook consuming LLM-driven skill updates
import { useAgentSkills } from '@/lib/workbench/agent-skills';

export function SkillPicker() {
  const { skills, loading, reload } = useAgentSkills();

  if (loading) return <Spinner />;

  return (
    <ul>
      {skills.map((s) => (
        <li key={s.id}>{skillDisplayLabel(s)}</li>
      ))}
    </ul>
  );
}

// Low-level LLM wrapper implementation
import { fetch } from '@/lib/fetch';

export async function callLLM(prompt: string, usageSource: string) {
  const res = await fetch('/api/llm', {
    method: 'POST',
    body: JSON.stringify({ prompt, usageSource }),
  });
  if (!res.ok) throw new Error('LLM call failed');
  return res.json(); // Returns { text: string, finishReason: string, … }
}

// Server-side agent decision endpoint
import { callLLM } from '@/lib/ai/llm';

export async function POST(req: Request) {
  const { sessionId, agentId } = await req.json();
  const prompt = buildAgentPrompt(sessionId, agentId);
  const llmResult = await callLLM(prompt, 'agentDecision');
  const skill = parseSkillFromText(llmResult.text);
  return new Response(JSON.stringify(skill), { status: 200 });
}

Error Resilience and Testing

The orchestration system includes comprehensive test coverage verifying LLM-centric behavior. According to tests/server/classroom-agent-mode.test.ts, fallback mechanisms ensure that when LLM calls fail, the system degrades gracefully to cached responses. Similarly, tests/web-search/claude.test.ts mocks the callLLM interface, confirming that production code depends on this centralized abstraction.

Summary

• OpenMAIC employs LLMs as the primary policy engine for agent decision-making through the centralized @/lib/ai/llm module.
• The callLLM function serves as the exclusive gateway for LLM interactions, invoked by API routes under /api/agent/* to generate skills, voice designs, and chat responses.
• React hooks like useAgentSkills synchronize UI states with LLM-generated outputs, rendering loading spinners during inference gaps.
• Error handling via AgentSkillsError and invalidateAgentSkills ensures the orchestration remains robust when providers are unavailable.
• Test files such as tests/workbench/agent-end-gap.test.ts explicitly verify the LLM-gap pattern, confirming the architecture's dependence on model inference for agent cognition.

Frequently Asked Questions

What function handles LLM calls in OpenMAIC?

The callLLM function exported from @/lib/ai/llm handles all LLM invocations. It accepts a prompt string and usage source identifier, then returns a structured response containing the generated text and finish reason.

How does OpenMAIC handle LLM failures during agent orchestration?

When callLLM encounters an error, the system activates fallback logic through AgentSkillsError and invalidateAgentSkills to serve cached skill lists or display error UI components, ensuring the classroom or workbench view remains functional.

Which test files demonstrate the LLM-dependent architecture?

tests/workbench/agent-end-gap.test.ts validates the "LLM-gap" UI pattern that appears during inference, while tests/server/classroom-agent-mode.test.ts verifies fallback behavior when LLM calls fail. Additionally, tests/web-search/claude.test.ts confirms the centrality of the callLLM abstraction by mocking its interface.

Can OpenMAIC work with multiple LLM providers?

Yes. The @/lib/ai/llm module abstracts provider-specific details, allowing callLLM to route requests to various backends including OpenAI, Anthropic, or other configured providers without changing the orchestration logic in the API routes.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}