How to Build Remotion Video Generation Plugins: A Complete Guide to OpenAI Codex Skills

Remotion video generation plugins are Codex-style "skills" that package reusable code snippets and best practices into a structured manifest, enabling AI-assisted creation of programmatic video content.

The openai/plugins repository hosts a reference implementation that demonstrates how to expose Remotion capabilities through the OpenAI Codex system. By following the established architecture in plugins/remotion/skills/remotion/, developers can create custom video generation workflows that the AI can reference for production-ready code.

Understanding the Remotion Plugin Architecture

The Remotion plugin follows a hierarchical structure that separates domain knowledge from executable code. This design allows the Codex runtime to surface relevant video generation patterns based on user intent.

Core Components

SKILL.md serves as the entry point in plugins/remotion/skills/remotion/SKILL.md. This manifest declares the skill name, description, and enumerates available rule files that contain specific implementation patterns.

Rule files (rules/*.md) live in plugins/remotion/skills/remotion/rules/ and contain self-contained tutorials for specific video tasks. Each file includes import statements, TypeScript code snippets, and CLI commands for tasks like animations, audio processing, FFmpeg operations, and Lottie integration.

The build system uses bun build.mts to package the skill into the .codex-plugin/ directory format. This script generates the plugin.json manifest that references the skills/ folder and prepares the package for distribution.

The final package structure follows this layout:

.codex-plugin/
  plugin.json          ← Generated manifest
  skills/
    remotion/
      SKILL.md
      rules/
        *.md

Step-by-Step Guide to Building a Remotion Plugin

Creating a new video generation skill requires scaffolding the project, authoring the manifest, and implementing specific rule files that address common Remotion development tasks.

1. Scaffold the Remotion Project

Begin by creating a minimal Remotion application using the official starter. This provides the baseline project structure for testing your plugin's code snippets:

npx create-video@latest --yes --blank --no-tailwind my-video

This command appears in the SKILL.md reference implementation (lines 14-18) as the standard initialization pattern.

2. Create the Plugin Skeleton

Within the repository, establish the plugin directory structure:

mkdir -p plugins/remotion/skills/remotion/rules
touch plugins/remotion/skills/remotion/SKILL.md

3. Author the SKILL.md Manifest

The manifest file uses YAML frontmatter to define metadata and Markdown to describe when to apply the skill. Reference the template in plugins/remotion/skills/remotion/SKILL.md and update the name and description fields. Include a bulleted list linking to each rule file you intend to ship.

4. Implement Rule Files

Each rule file in rules/*.md must contain:

• A descriptive heading and tags for searchability
• Required package installation commands (e.g., npx remotion add @remotion/lottie)
• Import statements and minimal functional code snippets
• Optional CLI usage examples (e.g., npx remotion render)

For example, a simple opacity animation rule in rules/animations.md would include:

import { interpolate, useCurrentFrame } from "remotion";

export const FadeIn = () => {
  const frame = useCurrentFrame();
  const opacity = interpolate(frame, [0, 30], [0, 1]);
  return <div style={{ opacity }}>Hello</div>;
};

The existing repository provides comprehensive examples in rules/lottie.md, rules/audio.md, and rules/transitions.md.

5. Reference Rules in the Manifest

Update SKILL.md to include relative links to each rule file, following the reference list pattern found at the bottom of the existing manifest.

6. Build the Plugin Package

Execute the build script to copy the rule files into the final Codex-plugin format:

bun build.mts

This process is documented in the Building section of plugins/remotion/README.md.

7. Test Locally

Install the generated plugin in a local Codex server according to the official OpenAI documentation. The plugin should surface under the "Remotion" skill set, with individual rules accessible to the AI assistant.

8. Publish

Commit the updated plugins/remotion/ directory to the main branch. The plugin becomes available to any Codex instance loading from the repository.

Code Examples and Implementation Details

SKILL.md Manifest Structure

A minimal skill manifest follows this format, as implemented in plugins/remotion/skills/remotion/SKILL.md:

---
name: my-remotion
description: Custom Remotion best-practice skill
metadata:
  tags: remotion video custom
---

## When to use

Use this skill when you need a tailored animation or audio effect in a Remotion project.

- [rules/animations.md](rules/animations.md) – Simple opacity animation
- [rules/audio.md](rules/audio.md) – Importing and trimming audio

Creating Rule Files for Video Effects

When adding new functionality, create specific rule files with complete implementation details. For example, rules/custom-effect.md might contain:

## Custom color-shift effect

**Tags:** remotion, effect, color

```tsx
import { interpolate, useCurrentFrame } from "remotion";

export const ColorShift = () => {
  const frame = useCurrentFrame();
  const hue = interpolate(frame, [0, 120], [0, 360]);
  return <div style={{ filter: `hue-rotate(${hue}deg)` }}>🌈</div>;
};

### Rendering and CLI Integration

The plugin should demonstrate how to use these components within a Remotion composition and render them:

```tsx
import { Composition } from "remotion";
import { ColorShift } from "./ColorShift";

export const MyVideo = () => (
  <Composition
    id="MyVideo"
    component={ColorShift}
    durationInFrames={120}
    fps={30}
    width={1080}
    height={1920}
  />
);

Execute the render using the standard CLI:

npx remotion render MyVideo out.mp4

These commands are derived from the existing rule files in rules/videos.md and rules/transitions.md.

Summary

• Remotion video generation plugins package code snippets and CLI commands into a Codex-compatible skill format located in plugins/remotion/skills/remotion/.
• The SKILL.md manifest acts as the entry point, while rule files (rules/*.md) contain specific implementation patterns for animations, audio, and video processing.
• Use bun build.mts to package the skill into the .codex-plugin/ directory structure for distribution.
• Each rule file should include complete import statements, functional code snippets, and installation commands to ensure AI-generated code is immediately executable.
• Testing occurs through local Codex server installation before publishing to the main branch of the repository.

Frequently Asked Questions

What is the difference between SKILL.md and rule files?

SKILL.md functions as the skill's entry point and catalog, declaring the name, description, and tags while listing available rule files. Rule files (rules/*.md) contain the actual implementation details—specific code snippets, package installation commands, and CLI usage examples for discrete tasks like Lottie animations or audio trimming.

How does the bun build.mts script work?

The build script processes the plugins/remotion/skills/ directory and copies the contents into a .codex-plugin/ folder, generating the required plugin.json manifest that declares the plugin's version and capabilities. This packaging step ensures the skill conforms to the Codex plugin specification for runtime discovery.

Can I use existing Remotion packages in my rules?

Yes. The reference implementation demonstrates integration with official Remotion packages such as @remotion/lottie and @remotion/media-utils. Include the installation command (e.g., npx remotion add @remotion/lottie) in your rule file, followed by the corresponding import statements and usage examples, as shown in plugins/remotion/skills/remotion/rules/lottie.md.

How do I test my plugin locally before publishing?

After running bun build.mts to generate the .codex-plugin/ directory, install the plugin in a local OpenAI Codex server instance following the official Codex plugin documentation. Verify that the skill appears in the Remotion category and that the AI can correctly reference individual rules when prompted for video generation tasks.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}