# How to Use Remotion for Programmatic Video Generation: Complete Guide with OpenAI Plugins

> Learn to create videos programmatically with Remotion and OpenAI plugins. This guide covers scaffolding, animation, audio, and FFmpeg rendering for seamless video generation.

- Repository: [OpenAI/plugins](https://github.com/openai/plugins)
- Tags: how-to-guide
- Published: 2026-07-04

---

**Remotion is a React-based framework that enables developers to generate video assets programmatically using JavaScript or TypeScript, with the OpenAI Plugins repository providing production-ready patterns for project scaffolding, animation, audio synthesis, and FFmpeg rendering.**

This guide explains how to use Remotion for programmatic video generation based on the comprehensive skill set defined in the OpenAI Plugins repository. The framework allows you to write ordinary React components that render to MP4 files, while the plugin's rule files encode best practices for everything from composition structure to data-driven visualizations.

## Project Setup and Composition Structure

To begin using Remotion for programmatic video generation, scaffold a new project using the official CLI. According to [`plugins/remotion/skills/remotion/SKILL.md`](https://github.com/openai/plugins/blob/main/plugins/remotion/skills/remotion/SKILL.md), the standard initialization command sets up the development environment with TypeScript support and the Remotion Studio for rapid preview.

```bash
npx create-remotion@latest my-video
cd my-video && npm install

```

Once scaffolded, define your video structure using the **Composition** component. As documented in [`plugins/remotion/skills/remotion/rules/compositions.md`](https://github.com/openai/plugins/blob/main/plugins/remotion/skills/remotion/rules/compositions.md), a composition acts as a React component that returns JSX describing the video timeline, including dimensions, frame rate, and duration.

```tsx
// src/Video.tsx
import {Composition} from 'remotion';
import {MyScene} from './scenes/MyScene';

export const RemotionRoot = () => (
  <>
    <Composition
      id="MyVideo"
      component={MyScene}
      durationInFrames={300}
      fps={30}
      width={1920}
      height={1080}
    />
  </>
);

```

## Asset Management and Animation

Load static assets using Remotion's built-in components. The [`plugins/remotion/skills/remotion/rules/assets.md`](https://github.com/openai/plugins/blob/main/plugins/remotion/skills/remotion/rules/assets.md) file specifies that images, video clips, and fonts should be placed in the `public/` directory and referenced via `staticFile()` to ensure proper bundling and pre-loading.

```tsx
// src/scenes/MyScene.tsx
import {Img, useCurrentFrame, interpolate, staticFile} from 'remotion';

export const MyScene = () => {
  const frame = useCurrentFrame();
  const opacity = interpolate(frame, [0, 30], [0, 1]);

  return <Img src={staticFile('assets/logo.png')} style={{opacity}} />;
};

```

For motion, Remotion provides **interpolation** and **physics-based spring** functions. The [`animations.md`](https://github.com/openai/plugins/blob/main/animations.md) rule file documents these as the primary methods for programmatic animation, accepting frame indices and configuration objects to calculate values for transforms, opacity, or positioning.

```tsx
// src/scenes/MyScene.tsx
import {spring, useCurrentFrame} from 'remotion';

export const MyScene = () => {
  const frame = useCurrentFrame();
  const scale = spring({frame, fps: 30, config: {damping: 200}});
  return (
    <div style={{fontSize: 80, transform: `scale(${scale})`}}>
      Hello, Remotion!
    </div>
  );
};

```

## Audio Integration and Voice Synthesis

Add background tracks using the `<Audio>` component, which supports trimming and volume control via props. The [`plugins/remotion/skills/remotion/rules/audio.md`](https://github.com/openai/plugins/blob/main/plugins/remotion/skills/remotion/rules/audio.md) file explains how to handle audio synchronization, while [`voiceover.md`](https://github.com/openai/plugins/blob/main/voiceover.md) provides patterns for integrating AI-generated speech.

To generate AI voice-overs, call external TTS APIs like ElevenLabs from a Node.js script, then reference the resulting files in your composition:

```typescript
// voiceover.ts
import fetch from 'node-fetch';
import fs from 'fs';

const text = "Welcome to our video!";
const resp = await fetch('https://api.elevenlabs.io/v1/text-to-speech', {
  /* auth and payload configuration */
});
const buffer = await resp.buffer();
fs.writeFileSync('public/voice.mp3', buffer);

```

Then consume the generated audio in your scene:

```tsx
// src/scenes/MyScene.tsx
import {Audio} from 'remotion';
import {staticFile} from 'remotion';

export const MyScene = () => (
  <>
    <Audio src={staticFile('voice.mp3')} startFrom={0} endAt={150} />
    <Audio src={staticFile('audio/bgm.mp3')} />
  </>
);

```

## Video Composition and Transitions

Insert external video clips using the `<Video>` component with controls for looping, trimming, and volume adjustment. The [`plugins/remotion/skills/remotion/rules/videos.md`](https://github.com/openai/plugins/blob/main/plugins/remotion/skills/remotion/rules/videos.md) file covers these patterns, noting that pitch-shifting requires server-side processing.

For scene transitions, use `TransitionSeries` from the transitions package. As outlined in [`transitions.md`](https://github.com/openai/plugins/blob/main/transitions.md), this utility enables fade effects, wipes, and overlays between compositions:

```tsx
// src/scenes/TransitionScene.tsx
import {TransitionSeries, Fade} from '@remotion/transition';
import {MyScene} from './MyScene';

export const TransitionScene = () => (
  <TransitionSeries>
    <Fade from={0} to={30}>
      <MyScene />
    </Fade>
  </TransitionSeries>
);

```

## Data Visualization and Charts

Render data-driven visualizations directly within your video pipeline. The [`plugins/remotion/skills/remotion/rules/charts.md`](https://github.com/openai/plugins/blob/main/plugins/remotion/skills/remotion/rules/charts.md) file provides utilities for bar, line, and pie charts that accept data arrays and render SVG graphics synchronized to the video timeline.

```tsx
// src/scenes/ChartScene.tsx
import {BarChart} from '@remotion/charts';

export const ChartScene = () => (
  <BarChart
    data={[{label: 'A', value: 30}, {label: 'B', value: 70}]}
    width={800}
    height={400}
  />
);

```

## Rendering and Export

Export the final video using the Remotion CLI, which handles FFmpeg integration automatically. The [`plugins/remotion/skills/remotion/rules/ffmpeg.md`](https://github.com/openai/plugins/blob/main/plugins/remotion/skills/remotion/rules/ffmpeg.md) file documents this final step, specifying that the command renders the React components to a video file using headless browser technology combined with FFmpeg encoding.

```bash
npx remotion render MyVideo out.mp4

```

This command processes all compositions, evaluates the JavaScript at each frame, and encodes the output using the FFmpeg configuration specified in your project.

## Summary

- **Remotion** enables programmatic video generation through React components that render to video files via FFmpeg integration
- The OpenAI Plugins repository provides structured guidance in `plugins/remotion/skills/remotion/` covering scaffolding, compositions, and rendering workflows
- Key utilities include `<Composition>` for timeline definition, `interpolate` and `spring` for frame-accurate animations, and `staticFile()` for asset management
- Audio workflows support both background music and AI-generated voiceovers via ElevenLabs API integration as documented in [`voiceover.md`](https://github.com/openai/plugins/blob/main/voiceover.md)
- Final output is generated using `npx remotion render` with automatic FFmpeg processing as specified in [`ffmpeg.md`](https://github.com/openai/plugins/blob/main/ffmpeg.md)

## Frequently Asked Questions

### Can Remotion be used for server-side video generation?

Yes, Remotion supports server-side rendering through Node.js environments. While the Remotion Studio provides a browser-based preview during development, the `npx remotion render` command executes server-side using FFmpeg. The [`voiceover.md`](https://github.com/openai/plugins/blob/main/voiceover.md) rule specifically demonstrates server-side API calls to ElevenLabs for generating audio assets before composition, and several audio processing features like pitch-shifting are documented as server-side only.

### How do I handle external media assets in Remotion?

Use the `staticFile()` function to reference assets in the `public/` directory, as detailed in [`plugins/remotion/skills/remotion/rules/assets.md`](https://github.com/openai/plugins/blob/main/plugins/remotion/skills/remotion/rules/assets.md). For images and video clips, import the `<Img>` and `<Video>` components from Remotion, which handle automatic pre-loading and frame-accurate rendering. All assets must be available at build time for reliable rendering in the final export.

### What animation utilities does Remotion provide for programmatic video?

Remotion provides `interpolate` for linear transitions between values and `spring` for physics-based animations. According to [`animations.md`](https://github.com/openai/plugins/blob/main/animations.md), these functions accept the current frame index and return calculated values for properties like opacity, transform, or position. The `useCurrentFrame()` hook gives components access to the timeline position for frame-accurate calculations.

### Can I generate data visualizations programmatically with Remotion?

Yes, the [`charts.md`](https://github.com/openai/plugins/blob/main/charts.md) rule file documents utilities for rendering bar charts, line graphs, and pie charts directly within compositions. These components accept data arrays and dimensions as props, rendering SVG-based graphics that animate smoothly with the video timeline. This approach is ideal for automated reporting videos or data-driven content pipelines.