# How the OpenMAIC Orchestration System Uses LLMs for Agent Decision-Making

> Discover how the OpenMAIC orchestration system leverages LLMs for intelligent agent decision-making. Learn about its innovative approach to generating actions, skills, and voice designs.

- Repository: [MAIC/OpenMAIC](https://github.com/THU-MAIC/OpenMAIC)
- Tags: deep-dive
- Published: 2026-09-11

---

**Yes, the OpenMAIC orchestration system uses Large Language Models (LLMs) as the primary decision-making engine, invoking the `callLLM` function from `@/lib/ai/llm` to generate agent actions, skills, and voice designs based on session context.**

The THU-MAIC/OpenMAIC repository implements an agentic framework where LLMs serve as the policy layer rather than hard-coded rules. When an agent needs to determine its next action—whether filling a thinking gap, generating a voice design, or resolving a classroom prompt—the orchestration layer calls the centralized LLM wrapper to synthesize decisions from contextual data stored in the session store.

## The LLM-Centric Architecture

### The Core Wrapper Module (`@/lib/ai/llm`)

At the heart of OpenMAIC's decision-making lies the `callLLM` function exported from `@/lib/ai/llm`. This module abstracts provider-specific implementations, allowing the orchestration layer to transmit prompts containing session state, agent roles, and user context to external providers like OpenAI or Anthropic.

### API Route Integration

Server-side decision entry points reside in `/api/agent/*` routes (e.g., *agent-skills* endpoints). When an agent requires a new decision, these routes construct prompts and invoke `callLLM` with usage source identifiers such as `'agentDecision'`, bridging HTTP requests with LLM inference.

## The 5-Step Decision Pipeline

OpenMAIC processes agent decisions through a structured pipeline that transforms LLM outputs into UI updates:

1. **Request Entry**: API routes under `/api/agent/*` receive requests when an agent needs a new decision.

2. **LLM Invocation**: The `@/lib/ai/llm` module's `callLLM` function transmits the constructed prompt to configured providers.

3. **Response Handling**: Workbench helpers like `skillDisplayLabel` and `skillTitle`, along with `parseSkillFromText`, extract structured data (skill handles, voice specifications) from the LLM's text output and persist them to the session store.

4. **UI Synchronization**: React hooks `useAgentSkills` and `useWorkspacePaneNavigation` trigger re-renders, displaying the generated decision in timelines or classroom views while handling loading states.

5. **Fallback Handling**: If `callLLM` fails, `AgentSkillsError` and `invalidateAgentSkills` logic serve cached skill lists or error UIs, ensuring the orchestration remains robust.

## The "LLM-Gap" Pattern and Thinking States

The LLM serves as the **decisive engine** for any "thinking" step. When the UI renders an "LLM-gap" row—as verified in [`tests/workbench/agent-end-gap.test.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/tests/workbench/agent-end-gap.test.ts)—this indicates the client has dispatched a request to `callLLM` and is awaiting the model's output to fill the timeline gap. This design makes the LLM the policy that determines the next agent action.

## Implementation Examples

The following code demonstrates the integration between React hooks, the LLM wrapper, and server-side orchestration:

```tsx
// React hook consuming LLM-driven skill updates
import { useAgentSkills } from '@/lib/workbench/agent-skills';

export function SkillPicker() {
  const { skills, loading, reload } = useAgentSkills();

  if (loading) return <Spinner />;

  return (
    <ul>
      {skills.map((s) => (
        <li key={s.id}>{skillDisplayLabel(s)}</li>
      ))}
    </ul>
  );
}

```

```ts
// Low-level LLM wrapper implementation
import { fetch } from '@/lib/fetch';

export async function callLLM(prompt: string, usageSource: string) {
  const res = await fetch('/api/llm', {
    method: 'POST',
    body: JSON.stringify({ prompt, usageSource }),
  });
  if (!res.ok) throw new Error('LLM call failed');
  return res.json(); // Returns { text: string, finishReason: string, … }
}

```

```ts
// Server-side agent decision endpoint
import { callLLM } from '@/lib/ai/llm';

export async function POST(req: Request) {
  const { sessionId, agentId } = await req.json();
  const prompt = buildAgentPrompt(sessionId, agentId);
  const llmResult = await callLLM(prompt, 'agentDecision');
  const skill = parseSkillFromText(llmResult.text);
  return new Response(JSON.stringify(skill), { status: 200 });
}

```

## Error Resilience and Testing

The orchestration system includes comprehensive test coverage verifying LLM-centric behavior. According to [`tests/server/classroom-agent-mode.test.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/tests/server/classroom-agent-mode.test.ts), fallback mechanisms ensure that when LLM calls fail, the system degrades gracefully to cached responses. Similarly, [`tests/web-search/claude.test.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/tests/web-search/claude.test.ts) mocks the `callLLM` interface, confirming that production code depends on this centralized abstraction.

## Summary

- OpenMAIC employs **LLMs as the primary policy engine** for agent decision-making through the centralized `@/lib/ai/llm` module.
- The **`callLLM`** function serves as the exclusive gateway for LLM interactions, invoked by API routes under `/api/agent/*` to generate skills, voice designs, and chat responses.
- **React hooks** like `useAgentSkills` synchronize UI states with LLM-generated outputs, rendering loading spinners during inference gaps.
- **Error handling** via `AgentSkillsError` and `invalidateAgentSkills` ensures the orchestration remains robust when providers are unavailable.
- **Test files** such as [`tests/workbench/agent-end-gap.test.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/tests/workbench/agent-end-gap.test.ts) explicitly verify the LLM-gap pattern, confirming the architecture's dependence on model inference for agent cognition.

## Frequently Asked Questions

### What function handles LLM calls in OpenMAIC?

The **`callLLM`** function exported from `@/lib/ai/llm` handles all LLM invocations. It accepts a prompt string and usage source identifier, then returns a structured response containing the generated text and finish reason.

### How does OpenMAIC handle LLM failures during agent orchestration?

When `callLLM` encounters an error, the system activates fallback logic through `AgentSkillsError` and `invalidateAgentSkills` to serve cached skill lists or display error UI components, ensuring the classroom or workbench view remains functional.

### Which test files demonstrate the LLM-dependent architecture?

[`tests/workbench/agent-end-gap.test.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/tests/workbench/agent-end-gap.test.ts) validates the "LLM-gap" UI pattern that appears during inference, while [`tests/server/classroom-agent-mode.test.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/tests/server/classroom-agent-mode.test.ts) verifies fallback behavior when LLM calls fail. Additionally, [`tests/web-search/claude.test.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/tests/web-search/claude.test.ts) confirms the centrality of the `callLLM` abstraction by mocking its interface.

### Can OpenMAIC work with multiple LLM providers?

Yes. The `@/lib/ai/llm` module abstracts provider-specific details, allowing `callLLM` to route requests to various backends including OpenAI, Anthropic, or other configured providers without changing the orchestration logic in the API routes.