How to Test a Codex Plugin Locally: Complete Setup and Execution Guide

Testing a Codex plugin locally requires installing the Codex CLI, copying the plugin bundle to ~/.codex/plugins/, configuring any MCP connectors, and executing skills with codex exec in an isolated sandbox.

Local testing is essential for verifying plugin behavior before publishing to the marketplace. The openai/plugins repository provides a reference implementation showing how to validate integrations against the exact runtime used in production. By following the Codex CLI workflow, you can test a Codex plugin locally using sandboxed execution and deterministic benchmarks.

Prerequisites and CLI Setup

Before testing any plugin, you must install the official Codex CLI and ensure it is available on your PATH. The CLI provides the codex exec and codex mcp commands required for local execution.

Clone the reference repository to access example plugins and test suites:

git clone https://github.com/openai/plugins.git && cd plugins

Installing the Plugin Locally

Local plugins reside in the ~/.codex/plugins/ directory. To install a plugin for testing, copy the plugin folder from your working repository into this location.

For example, to install the Remotion plugin locally:

mkdir -p ~/.codex/plugins
cp -r plugins/remotion ~/.codex/plugins/remotion

Each plugin must contain a plugin manifest at .codex-plugin/plugin.json that declares the plugin name, version, and skill entry points. In the Remotion example, this file is located at plugins/remotion/.codex-plugin/plugin.json and defines the directory structure the Codex runtime expects.

Configuring MCP Connectors

Plugins that interact with external APIs require Managed-Connector-Provider (MCP) configurations. Use the codex mcp add command to register these services before execution.

For example, the Render plugin requires an MCP connector with authentication:

codex mcp add render \
   --url https://mcp.render.com/mcp \
   --bearer-token-env-var RENDER_API_KEY

This command wires the connector to your local Codex environment, allowing the plugin to access external resources during testing. The syntax is documented in the skill definition at plugins/render/skills/render-mcp/SKILL.md.

Executing Skills with codex exec

The codex exec command runs a skill in an isolated temporary workspace that mirrors the production marketplace runtime. This is the canonical method for end-to-end testing.

Execute a specific skill by providing the plugin path, skill directory, and request:

codex exec \
   --plugin ~/.codex/plugins/remotion \
   --skill skills/video-generator \
   --request "Create a 5-second clip of a cat playing piano"

This invocation processes the request through the exact pipeline used when users call the plugin from the marketplace, including sandboxing and resource limits.

Benchmarking with plugin-eval

For deterministic validation and performance analysis, use the plugin-eval harness. This tool runs standardized benchmarks and generates token-usage reports.

Run a benchmark suite defined in the plugin's configuration:

plugin-eval benchmark ~/.codex/plugins/remotion \
   --config ~/.codex/plugins/remotion/.plugin-eval/benchmark.json

The evaluation framework is documented in plugin-eval/README.md within the repository. This step captures execution logs and measures resource consumption before publication.

Running Unit Tests

Many plugins ship with Node.js test suites that validate API integrations and skill logic. These tests run independently of the Codex sandbox and verify individual components.

Execute the OpenAI-Developers plugin test suite:

node --test plugins/openai-developers/tests/openai-platform-api-key.test.mjs

Unit tests are typically documented in the plugin's README, such as plugins/openai-developers/README.md, and provide fast feedback during development iterations.

Iteration Workflow

When modifying plugin behavior, edit the skill definition—usually located at skills/<skill-name>/SKILL.md—then re-run the execution or test commands. This cycle ensures changes are validated against both the sandboxed runtime and unit-level assertions before committing.

Summary

• Install locally by copying the plugin to ~/.codex/plugins/ and verifying the .codex-plugin/plugin.json manifest exists.
• Configure MCPs using codex mcp add to enable external API access during testing.
• Execute skills with codex exec to validate behavior in the production-equivalent sandbox.
• Benchmark performance using plugin-eval benchmark to capture token usage and execution logs.
• Run unit tests with node --test to verify specific API integrations and logic paths.

Frequently Asked Questions

Where does the Codex CLI look for local plugins?

The CLI searches the ~/.codex/plugins/<plugin-name> directory for plugin bundles. Each bundle must contain a .codex-plugin/plugin.json manifest that defines the plugin metadata and skill entry points, as seen in the Remotion example at plugins/remotion/.codex-plugin/plugin.json.

What is the purpose of the plugin.json manifest?

The plugin.json file declares the plugin's identity, version, and directory structure to the Codex runtime. It tells the CLI where skill definitions reside and which files constitute the plugin bundle, enabling the codex exec command to locate and invoke the correct handlers.

How do I handle external API dependencies when testing locally?

Use the codex mcp add command to register Managed-Connector-Providers for external services. For example, codex mcp add render --url https://mcp.render.com/mcp --bearer-token-env-var RENDER_API_KEY configures the Render connector, allowing the plugin to authenticate and communicate with third-party APIs during local execution.

Can I run automated benchmarks before publishing?

Yes. The plugin-eval tool provides a deterministic benchmark harness. Run plugin-eval benchmark <plugin-path> --config <path-to-benchmark.json> to generate token-usage reports and execution logs, ensuring the plugin meets performance standards before marketplace submission as documented in plugin-eval/README.md.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}