MCP Server Integration and AI Agent Interaction with OpenCut: Implementation Guide

OpenCut's monorepo architecture enables AI agents to control the video editor programmatically through a dedicated MCP (Multi-Channel Processor) server that bridges REST API endpoints with the React-based front-end interface.

OpenCut is organized as a monorepo that cleanly separates the front-end editor (the Web app) from the back-end services (the API app). This architectural separation makes MCP server integration straightforward, allowing AI agents to send editing commands via HTTP endpoints that interface with the core editor functionality. By leveraging the existing Elysia-based API layer and Cloudflare Worker runtime, developers can extend OpenCut with programmatic AI control without compromising the modular design.

Understanding the OpenCut Architecture

The codebase is divided into two primary layers that communicate via HTTP, creating a natural integration point for external AI agents.

Web Layer (Editor UI)

The front-end application lives in apps/web/src/routes/__root.tsx, which serves as the primary entry point for the React-based editing interface. Built with Vite, Tailwind CSS, and TypeScript, this layer handles user interactions and communicates with the back-end using fetch utilities defined in apps/web/src/lib/utils.ts. The web client runs independently on its own development server (typically localhost:5173), making it accessible for MCP commands forwarded from the API layer.

API Layer (Service Backend)

The back-end service resides in apps/api/src/index.ts and uses Elysia, a lightweight TypeScript web framework optimized for edge computing. The API is configured with CloudflareAdapter (lines 2-4), enabling deployment as a Cloudflare Worker with Ahead-of-Time (AoT) compilation via .compile() (lines 14-15). This runtime environment provides the ideal foundation for hosting MCP endpoints that need to handle AI agent traffic with low latency and automatic scaling.

Implementing MCP Server Endpoints

Adding MCP capabilities requires extending the existing API routes to accept and process AI-generated commands.

Creating the /mcp/command Route

New MCP functionality can be added by defining additional routes in apps/api/src/index.ts. Following the pattern established by the existing /echo endpoint, an MCP handler validates incoming JSON payloads and dispatches them to the appropriate editor functions:

// apps/api/src/index.ts
import { Elysia, t } from "elysia";
import { CloudflareAdapter } from "elysia/adapter/cloudflare-worker";

export default new Elysia({ adapter: CloudflareAdapter })
  .post(
    "/mcp/command",
    async ({ body }) => {
      const { action, params } = body;
      if (action === "trim") {
        const resp = await fetch("http://localhost:5173/api/editor/trim", {
          method: "POST",
          headers: { "Content-Type": "application/json" },
          body: JSON.stringify(params),
        });
        return resp.json();
      }
      return { error: "Unsupported action" };
    },
    {
      body: t.Object({
        action: t.String(),
        params: t.Record(t.String(), t.Any()),
      }),
    }
  )
  .compile();

Validation and Command Dispatch

The Elysia framework provides runtime type validation through the t validator. Each MCP command must specify its action type and parameters, ensuring that AI agents transmit correctly structured data before the system attempts execution. Validated commands are then dispatched either via HTTP fetch calls to the web layer or through WebSocket connections for real-time operations.

AI Agent Communication Flow

The interaction between AI agents and the OpenCut editor follows a structured request-response pattern that mirrors standard REST conventions.

Request Structure and Endpoints

AI agents communicate by sending JSON payloads to the /mcp/command endpoint. A typical request includes the action name and specific parameters required for video manipulation:

import requests

payload = {
    "action": "trim",
    "params": {
        "clipId": "abc123",
        "start": 5.0,
        "end": 12.0
    }
}

resp = requests.post(
    "https://api.opencut.app/mcp/command",
    json=payload,
    timeout=10
)

print(resp.json())

This closed-loop workflow enables agents to verify command execution and retrieve resulting state changes or error messages.

WebSocket vs HTTP Integration

For operations requiring real-time feedback, the MCP server can establish WebSocket connections to the front-end editor. This approach reduces latency for rapid successive commands (such as frame-by-frame adjustments) and allows the AI agent to receive progress updates during long-running operations like video export or effect rendering.

Front-end Integration with React Hooks

The editor UI receives MCP commands through dedicated React hooks that bridge the gap between the API layer and the component state.

The useMcp Hook Implementation

A custom hook in apps/web/src/hooks/useMcp.ts manages the connection state and message handling between the editor and the MCP server:

// apps/web/src/hooks/useMcp.ts
import { useEffect, useState } from "react";

export function useMcp() {
  const [status, setStatus] = useState<string>("idle");

  useEffect(() => {
    const ws = new WebSocket("ws://localhost:5173/mcp");
    ws.onmessage = (ev) => {
      const data = JSON.parse(ev.data);
      setStatus(data.status);
    };
    return () => ws.close();
  }, []);

  return status;
}

This hook can be extended to parse specific command types (such as trim, splice, or add_effects) and invoke the corresponding timeline manipulation functions exposed by the editor's core library.

Extending via Plugin Architecture

OpenCut employs a plugin-first architecture (referenced in README lines 21-22) that allows MCP functionality to be packaged as modular extensions rather than core modifications.

Plugins can register new MCP commands by exposing additional routes under /mcp/<plugin-name> without altering the main API entry point. This modularity ensures that AI-driven features—such as automated color correction or intelligent clip sequencing—can be developed independently and loaded dynamically based on user configuration.

Development Workflow and Security

Setting up the MCP integration requires configuring both the front-end and API development environments.

1. Install tooling: Run proto use && bun install (as specified in README lines 32-43) to install the monorepo dependencies.
2. Start the API: Execute moon run api:dev to launch the Elysia server on localhost:8787, including any MCP routes defined in apps/api/src/index.ts.
3. Iterate: Modify the API handlers and test AI agent interactions using the local development endpoints.

Because the API runs within the Cloudflare Worker sandbox, MCP endpoints inherit the same security model—including per-request isolation, automatic rate limiting, and DDoS protection—without additional configuration.

Summary

• OpenCut's monorepo structure separates the React front-end (apps/web) from the Elysia API (apps/api), creating clear integration points for MCP servers.
• AI agents interact with the editor by posting JSON commands to /mcp/command endpoints that validate requests and forward them to the web layer.
• The Cloudflare Worker runtime provides a secure, scalable foundation for hosting MCP endpoints with AoT compilation.
• Real-time communication can be implemented via WebSocket connections managed through React hooks like useMcp.
• Plugin architecture allows MCP capabilities to be extended modularly without modifying core editor code.

Frequently Asked Questions

What is an MCP server in the context of OpenCut?

An MCP (Multi-Channel Processor) server in OpenCut is a specialized API endpoint layer that receives commands from AI agents and translates them into editor actions. Implemented as an extension to the existing Elysia-based API in apps/api/src/index.ts, it acts as a bridge between external AI systems and the React-based video editing interface, enabling programmatic control over timeline manipulation, effects application, and export operations.

How do AI agents authenticate with the OpenCut API?

The source analysis indicates that the API runs on Cloudflare Workers, which provide built-in request sandboxing and security controls. While specific authentication implementations aren't detailed in the core files, the Elysia framework supports middleware injection for API key validation or OAuth token verification within the route handlers defined in apps/api/src/index.ts, allowing developers to secure /mcp/command endpoints appropriately.

Can MCP commands trigger real-time updates in the editor UI?

Yes, real-time updates are supported through WebSocket connections established via the useMcp hook pattern in the front-end. When an AI agent sends a command to the MCP server, the API can broadcast the instruction through a WebSocket channel to apps/web/src/hooks/useMcp.ts, which then updates the React component state and reflects changes immediately in the timeline view without requiring a page refresh.

What technologies power the OpenCut MCP integration?

The integration leverages Elysia (a TypeScript web framework) for the API layer, Cloudflare Workers for serverless deployment, React with TypeScript for the front-end, and Bun as the JavaScript runtime. The monorepo is managed using Moon (the task runner), with type validation handled by Elysia's built-in typebox validators, ensuring end-to-end type safety between AI agents and the editor interface.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}