Agent Runtime in OpenMAIC's Pro Workbench: Core Execution Engine Explained

The agent runtime is the central server-side execution engine that powers all AI-driven capabilities in OpenMAIC's Pro workbench, handling session orchestration, tool validation, LLM invocation, and state synchronization.

The Pro workbench in OpenMAIC represents the advanced authoring environment where users leverage intelligent agents for content generation. At its foundation lies the agent runtime—a dedicated server-side subsystem that bridges interactive UI components with backend AI services. This article examines how the runtime operates, its architectural responsibilities, and how developers can interact with it.

What the Agent Runtime Does in OpenMAIC Pro

The agent runtime lives in lib/server/agent-runtime/* and serves four critical functions for the Pro workbench:

• Session lifecycle management – creates, persists, and streams conversation state
• Tool registry and validation – exposes callable tools with contract enforcement
• AI service orchestration – invokes LLMs, image/video generators, and voice cloning
• UI state synchronization – broadcasts runtime status and diagnostics to the client

When a user accesses Pro features—such as slide generation, voice-over addition, or skill execution—the workbench queries the runtime to determine availability and dispatch operations.

Session Orchestration and State Management

The runtime's entry point, runner.ts, handles all session-level operations. It registers new sessions, loads historical context, and establishes event streams for real-time UI updates.

// lib/server/agent-runtime/runner.ts
// Core session orchestration logic

Each session receives a stable sessionId that the Pro workbench uses to track ongoing operations. The runner coordinates between:

1. Incoming HTTP requests from owner-session-client.ts
2. Tool call dispatch to specialized modules like generation-tools.ts
3. Event streaming back to the UI via event-notify-bus.ts

The owner-session-client (lib/workbench/owner-session-client.ts) provides the client-side API surface that the workbench uses to fetch active sessions, check status, and retrieve runtime-driven URLs.

Tool Registry and Call Validation

Before any AI operation executes, the runtime validates tool calls through tool-call-integrity.ts. This ensures that:

• Tool names match registered contracts
• Arguments conform to expected schemas
• Execution context carries proper authorization

// Example: Invoking a generation tool from a Pro session
import { GENERATION_TOOL_NAMES } from '@/lib/server/agent-runtime/generation-tools';
import { callAgentTool } from '@/lib/server/agent-runtime/runner';

async function generateSlide(deckId: string) {
  const result = await callAgentTool({
    tool: GENERATION_TOOL_NAMES.SLIDE_GENERATOR,
    args: { deckId, prompt: 'Explain photosynthesis' },
  });
  // result contains generated slide content, ready for UI display
  return result;
}

Available tools include:

• SLIDE_GENERATOR – AI-powered presentation creation
• IMAGE_GENERATOR – Visual asset synthesis via generate-image.ts
• VIDEO_GENERATOR – Video composition via generate-video.ts
• VOICE_CLONE – Audio synthesis and voice replication
• MATERIAL_TOOLS – Asset upload and management utilities

Runtime Configuration and Feature Flags

The runtime's behavior is controlled by config.ts, which parses environment variables and produces a RuntimeConfig object:

// lib/server/agent-runtime/config.ts
// Environment-driven configuration

Key configuration sources:

• OPENMAIC_AGENT_RUNTIME_ENABLED – master toggle for runtime availability
• DATABASE_URL – persistence layer connection for session storage

// Example: Checking if the runtime is enabled before showing a Pro‑only button
import { getRuntimeConfig } from '@/lib/server/agent-runtime/config';

async function ProButton() {
  const cfg = await getRuntimeConfig(); // → { enabled: boolean, ... }
  if (!cfg.enabled) return null; // hide button if runtime is off
  return <button onClick={startProFeature}>Run Pro Feature</button>;
}

When disabled or misconfigured, the Pro workbench gracefully degrades—UI elements show disabled states while the server logs diagnostic warnings (validated in server/config-validation.test.ts).

AI Generation and Media Synthesis

The runtime delegates actual AI operations to specialized modules:

Module	Responsibility
generation-ai-call.ts	LLM invocation and response handling
generate-image.ts	Image synthesis pipeline
generate-video.ts	Video generation and composition
session-material.ts	Asset metadata and upload status

These modules abstract provider-specific implementations, presenting a unified interface to the runner regardless of underlying AI service.

State Synchronization and Diagnostics

Runtime state propagates to the UI through several channels:

• session-material.ts and session-urls.ts – expose asset URLs and upload progress
• event-notify-bus.ts – streams real-time status updates
• Video export integration – embeds runtime diagnostics into final outputs

// Example: Adding a runtime‑diagnostic to a video manifest (used internally)
import { addRuntimeDiagnostic } from '@/lib/video-export/emit';

function handleRuntimeError(err: Error) {
  addRuntimeDiagnostic({
    code: 'interactive-runtime-failure',
    message: err.message,
  });
}

This design ensures that failures are visible in exported content without breaking the user-facing output.

Pro Workbench Integration Points

The runtime connects to the Pro workbench UI through several established patterns:

use-workbench-pro-edit.ts – React hook that adapts editor layout based on runtime state, enabling dynamic tool palettes and context-sensitive controls.

pro-swap.ts – Handles seamless transitions between classic and Pro workbench modes, preserving session continuity and runtime badges.

tool-presentation.test.ts – Test suite verifying that every registered runtime tool surfaces correctly in the Pro UI, preventing drift between backend capabilities and frontend exposure.

Summary

The agent runtime in OpenMAIC's Pro workbench serves as the authoritative execution layer for all AI-driven authoring features:

• Lives server-side in lib/server/agent-runtime/* with runner.ts as the primary entry point
• Validates and dispatches tool calls through tool-call-integrity.ts and specialized tool modules
• Configures via environment variables parsed by config.ts
• Synchronizes state to the Pro workbench UI through owner-session-client.ts and material/session modules
• Gracefully degrades when disabled, ensuring consistent user experience

Frequently Asked Questions

What happens if the agent runtime is disabled in OpenMAIC?

The Pro workbench detects the disabled state via getRuntimeConfig() and hides Pro-only features. UI buttons and menus do not render, and the server logs configuration warnings without throwing errors. Users retain access to classic workbench functionality.

How does the agent runtime validate tool calls?

Validation occurs in tool-call-integrity.ts, which checks tool names against registered contracts and verifies argument schemas before execution. Invalid calls are rejected with diagnostic codes that propagate through event-notify-bus.ts to the UI.

Can I add custom tools to the OpenMAIC agent runtime?

Yes. Register new tool names in a dedicated *-tools.ts module following the pattern of generation-tools.ts, implement the execution logic, and export through runner.ts. The test suite in tool-presentation.test.ts ensures automatic UI exposure.

Where does the agent runtime store session data?

Session persistence uses the database connection specified by DATABASE_URL. The runner.ts module loads historical context at session start and commits state changes through the standard ORM layer, with session-material.ts handling asset-specific metadata.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}