# OpenMAIC Agent Tools for Planning and Building: Complete Capabilities Guide

> Explore OpenMAIC agent tools capabilities for planning and building dynamic educational content. Discover type-safe APIs for web search, voice synthesis, and more.

- Repository: [MAIC/OpenMAIC](https://github.com/THU-MAIC/OpenMAIC)
- Tags: capabilities-guide
- Published: 2026-09-12

---

**OpenMAIC agent tools provide type-safe APIs for course page generation, web search, voice synthesis, material handling, and curriculum management, enabling AI models to plan and build educational content dynamically.**

OpenMAIC is an open-source educational content authoring platform that exposes a rich **agent-runtime** for AI-driven course creation. The agent tools defined in this runtime allow large language models to invoke concrete functions—ranging from slide generation to voice cloning—while maintaining strict type safety and deterministic execution. These capabilities transform the model from a passive text generator into an active builder that can structure curricula, fetch external knowledge, and produce multimedia assets.

## Course Page Generation and Management

The primary planning capabilities in OpenMAIC center on **course page creation**, implemented in [`lib/server/agent-runtime/generation-tools.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/server/agent-runtime/generation-tools.ts). These tools enable the agent to scaffold entire learning units without manual intervention.

### Core Generation Functions

- **`generate_scene`** – Creates a new page (slide, quiz, interactive, or PBL module) by persisting a fully-rendered structure to the database. The agent supplies parameters like `title`, `brief`, `type`, and optional widget configurations.
- **`list_scenes`** – Returns the current course structure, allowing the agent to perform planning reasoning based on existing content.
- **`generate_actions`** – Regenerates the interactive timeline for a page, including optional text-to-speech (TTS) narration when `synthesizeAudio` is enabled.
- **`duplicate_scene`** – Clones an existing page without its actions, enabling rapid scaffolding of similar content.

This workflow allows the agent to **plan a new learning unit** by first describing the pedagogical intent, then calling `generate_scene` to materialize the page, and finally using `generate_actions` to flesh out interactivity.

## Web Search and Knowledge Acquisition

When the agent encounters facts it cannot verify internally—such as recent statistics or product specifications—it invokes the **`web_search`** tool defined in [`lib/server/agent-runtime/web-search.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/server/agent-runtime/web-search.ts).

This tool performs an internet search and returns top results formatted as contextual text. Crucially, every discovered URL is registered in the session-wide URL store, guaranteeing **provenance for citations** and ensuring the agent can reference authoritative sources in generated content. The tool only registers when `resolveWebSearchCapability()` detects a configured provider, preventing invocation errors in offline deployments.

## Voice Synthesis and Cloning

For courses requiring a **personalized narrator**, OpenMAIC exposes voice cloning capabilities through [`lib/server/agent-runtime/voice-clone-tools.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/server/agent-runtime/voice-clone-tools.ts). These tools integrate with TTS pipelines to create consistent, branded audio experiences.

- **`clip_audio`** – Extracts a short sample (e.g., 5 seconds) from an existing user-provided audio file.
- **`register_voice`** – Registers the clipped sample as a new synthetic voice with the provider.

Once registered via `register_voice`, the voice becomes available to the `synthesizeSceneNarration` function during `generate_actions` calls, allowing the agent to generate narration using the cloned voice rather than generic TTS models.

## Material Handling and Asset Management

Educational content requires rich media assets. The agent interacts with binary materials through functions defined in [`lib/server/agent-runtime/material-media.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/server/agent-runtime/material-media.ts) and [`lib/server/agent-runtime/material-tools.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/server/agent-runtime/material-tools.ts).

These tools—such as `upload_material` and `list_materials`—allow the agent to **store, retrieve, and manage binary assets** including images, videos, and PDFs. During the building phase, the agent can embed these materials into generated pages or reference external resources previously uploaded by users, ensuring all necessary media is properly linked to the session.

## Curriculum and Roster Management

To support adaptive learning scenarios, OpenMAIC provides curriculum-level controls through [`lib/server/agent-runtime/roster-tools.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/server/agent-runtime/roster-tools.ts).

- **`set_roster`** and **`list_roster`** – Define which agents (students or teachers) are present in the session, enabling content personalization based on participant roles.
- **Curriculum allow-list tools** – Enforce course-level policies and constraints, ensuring the agent respects pedagogical guardrails while planning content.

By adjusting the roster dynamically, the agent can tailor difficulty levels and content types to specific learners without hard-coding demographic logic.

## Common Architecture Patterns

All OpenMAIC agent tools follow a **consistent implementation pattern** that ensures safety and reproducibility:

1. **Schema Definition** – Each tool declares its parameters using JSON Schema (via `typebox`), guaranteeing well-typed inputs and preventing malformed calls.
2. **Execution Function** – The `execute` method performs the actual work, whether database mutations, HTTP calls, or external AI service invocations.
3. **Checkpoint Emission** – After successful mutations, tools invoke `deps.onCheckpoint` to record deterministic events. This enables replay for testing or debugging and maintains audit trails.

Additionally, tools are **registered conditionally** based on deployment capabilities. For example, `web_search` only appears when a search provider is configured. This design gives the model **planning awareness**—it can query available tools and decide whether to invoke them, adapt its strategy based on infrastructure constraints, and avoid procedural failures.

## Practical Usage Examples

Below are representative code patterns showing how an agent invokes these tools during the course building workflow:

```typescript
// Generate a new slide page with structured content
await callTool('generate_scene', {
  stageId: 'stage-abc',
  order: 3,
  title: 'Newton’s Laws',
  type: 'slide',
  brief: 'Introduce the three laws of motion',
  materialFacts: ['Inertia', 'F=ma'],
});

```

```typescript
// Regenerate interactive actions with AI narration
await callTool('generate_actions', {
  stageId: 'stage-abc',
  order: 3,
  synthesizeAudio: true,
});

```

```typescript
// Clone an existing page to reuse its layout
await callTool('duplicate_scene', {
  stageId: 'stage-abc',
  templateOrder: 3,
  targetOrder: 5,
  title: 'Newton’s Laws – Review',
});

```

```typescript
// Fetch recent statistics to verify course content
await callTool('web_search', { 
  query: '2024 global renewable energy capacity' 
});

```

```typescript
// Create a custom voice for course narration
await callTool('clip_audio', { 
  source: '/uploads/user.wav', 
  startMs: 0, 
  endMs: 5000 
});
await callTool('register_voice', { 
  name: 'my-voice', 
  clipId: 'clip-123' 
});

```

*Note: The `callTool` abstraction represents the internal dispatch mechanism within the `@earendil-works/pi-agent-core` runtime.*

## Summary

- **OpenMAIC agent tools** expose concrete, type-safe functions that transform AI models into active course builders.
- **Page generation tools** (`generate_scene`, `duplicate_scene`, etc.) enable dynamic curriculum structure creation without manual layout work.
- **Web search capabilities** provide real-time knowledge acquisition with automatic URL provenance tracking.
- **Voice cloning tools** support personalized narration through `clip_audio` and `register_voice` integration.
- **Conditional registration** ensures agents only invoke available capabilities, enabling adaptive planning based on deployment configuration.
- **Checkpoint emission** guarantees deterministic execution and full auditability of the building process.

## Frequently Asked Questions

### What are OpenMAIC agent tools?

OpenMAIC agent tools are type-safe API functions defined in the agent-runtime (located in `lib/server/agent-runtime/`) that AI models invoke during course authoring. They provide concrete capabilities—such as creating slides, searching the web, or cloning voices—that allow the model to plan and build educational content structurally rather than generating text alone.

### How does the web_search tool support course planning?

The `web_search` tool enables the agent to fetch up-to-date information from the internet when internal knowledge is insufficient or outdated. According to the implementation in [`lib/server/agent-runtime/web-search.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/server/agent-runtime/web-search.ts), it returns formatted search results and registers all discovered URLs in a session-wide store, ensuring the agent can cite authoritative sources while building factually accurate content.

### Can OpenMAIC agent tools clone voices for narration?

Yes. Through [`lib/server/agent-runtime/voice-clone-tools.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/server/agent-runtime/voice-clone-tools.ts), the agent can extract audio samples using `clip_audio` and register them as synthetic voices via `register_voice`. These custom voices are then available to the TTS pipeline during `generate_actions` calls, allowing the agent to generate personalized narration for course pages using the instructor's or a branded voice.

### How does the tool registration system ensure reliability?

Tools in OpenMAIC are registered conditionally based on deployment capabilities—for example, `web_search` only appears when a provider is configured. This pattern, implemented across the runtime, allows the agent to query available tools before invocation and prevents runtime errors. Additionally, the [`tool-call-integrity.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/tool-call-integrity.ts) module guarantees that every tool call receives a matching result, preventing orphaned or incomplete operations during the building workflow.