How Harnesses Discover Plugins in the OpenAI Plugins Repository: A Data-Driven Guide

Harnesses discover plugins in the openai/plugins monorepo by parsing the marketplace index at .agents/plugins/marketplace.json, loading individual plugin manifests from .codex-plugin/plugin.json, and indexing skill capabilities through SKILL.md front-matter metadata.

The openai/plugins repository implements a declarative, filesystem-based discovery mechanism that requires no runtime code execution. Evaluation harnesses—such as the one used by plugin-eval—enumerate available capabilities through a deterministic three-stage process that walks static metadata files and Markdown front-matter.

The Three-Stage Plugin Discovery Process

Plugin discovery follows a hierarchical data flow that moves from the global marketplace index down to individual skill definitions. This architecture allows harnesses to build a complete capability index without importing or executing plugin code.

Stage 1: Locating the Marketplace Index

The harness begins by locating the marketplace index at .agents/plugins/marketplace.json. According to the repository README, this JSON file serves as the central registry that maps plugin identifiers to their source locations.

The marketplace file contains a flat map where each entry points to a plugin directory relative to the repository root or the user’s home directory (via ~/.agents/plugins/marketplace.json). Harnesses parse this file first to determine which plugins are available for evaluation.

Stage 2: Reading the Plugin Manifest

For each entry resolved from the marketplace, the harness reads the plugin manifest located at .codex-plugin/plugin.json within the plugin directory. As documented in the plugin-eval README, this manifest supplies critical metadata including the plugin name, version, and the interface.defaultPrompt configuration.

The manifest also specifies the pluginRoot, which the harness uses to resolve relative paths when searching for skill definitions in the next stage.

Stage 3: Skill Discovery via Front-Matter

Individual capabilities are discovered by parsing SKILL.md files located within the plugin's skills/ subdirectories. Each skill definition uses Markdown front-matter to declare its metadata, including retrieval.aliases, intents, entities, pathPatterns, and bashPatterns.

According to the Vercel plugin documentation, Codex uses this front-matter to match user intents to the correct skill without requiring runtime hooks. The harness walks the directory tree, extracts the YAML front-matter from each SKILL.md, and builds a searchable index of available capabilities.

Implementing Plugin Discovery in Python

The following implementation reproduces the discovery logic used by harnesses in the openai/plugins repository:

import json
import re
from pathlib import Path
import yaml

# Stage 1: Load the marketplace index

marketplace_path = Path('.agents/plugins/marketplace.json')
marketplace = json.loads(marketplace_path.read_text())

# Stage 2: Resolve plugin directories and read manifests

plugins = {}
for name, entry in marketplace.items():
    plugin_dir = Path(entry['source']['path'])
    manifest_path = plugin_dir / '.codex-plugin/plugin.json'
    manifest = json.loads(manifest_path.read_text())
    plugins[name] = manifest

# Stage 3: Build skill index from SKILL.md front-matter

def load_skill_meta(skill_dir: Path) -> dict:
    """Extract YAML front-matter from SKILL.md files."""
    text = (skill_dir / 'SKILL.md').read_text()
    fm = re.search(r'^---\n(.*?)\n---', text, re.S)
    return yaml.safe_load(fm.group(1)) if fm else {}

skill_index = {}
for name, manifest in plugins.items():
    skills_root = Path(manifest['pluginRoot']) / 'skills'
    for skill_path in skills_root.glob('**/SKILL.md'):
        meta = load_skill_meta(skill_path.parent)
        skill_index[skill_path.parent.name] = meta

This code demonstrates the data-driven approach: JSON parsing for structure, Markdown front-matter extraction for capability metadata, and filesystem walking for enumeration.

Key Files in the Discovery Pipeline

Several files serve as the backbone of the discovery mechanism:

• .agents/plugins/marketplace.json — The global marketplace index that maps plugin identifiers to their relative or absolute paths.
• plugins/plugin-eval/README.md — Documents the marketplace-based discovery mechanism and the relationship between the marketplace index and plugin manifests.
• */.codex-plugin/plugin.json — Per-plugin manifests containing version information, interface definitions, and the root directory pointer.
• */skills/**/SKILL.md — Skill definitions containing YAML front-matter for Codex-native intent matching.
• plugins/vercel/README.md — Provides examples of front-matter metadata structures including retrieval.aliases and pathPatterns.

Summary

• Discovery starts at the marketplace: Harnesses locate .agents/plugins/marketplace.json to enumerate available plugins.
• Manifests provide structure: Each plugin's .codex-plugin/plugin.json supplies metadata and file locations.
• Skills are data-driven: Capability detection relies on parsing YAML front-matter from SKILL.md files rather than executing code.
• Two installation contexts: Harnesses discover plugins from both the repository workspace and user home directories (~/.agents/plugins/).
• No runtime hooks required: The entire process operates through JSON and Markdown parsing, making it deterministic and safe.

Frequently Asked Questions

What is the role of marketplace.json in plugin discovery?

The marketplace.json file acts as the central registry that maps plugin names to their filesystem locations. Harnesses read this file first to determine which plugins to load, supporting both repository-local paths and user-specific installations in the home directory.

How does a harness identify individual skills within a plugin?

After loading a plugin manifest, the harness walks the skills/ subdirectory tree and parses the YAML front-matter from each SKILL.md file. This front-matter contains intents, entities, and pathPatterns that define how Codex matches user requests to specific capabilities.

Is code execution required for plugin discovery in openai/plugins?

No. The discovery process is completely data-driven and requires only JSON parsing and Markdown front-matter extraction. As noted in the source documentation, "no hooks are required" because all capability metadata is declared statically in configuration files.

Can harnesses discover plugins installed outside the repository?

Yes. The marketplace supports entries pointing to directories outside the repository root, including paths in the user's home directory via ~/.agents/plugins/marketplace.json. This allows harnesses to evaluate plugins installed globally or in separate workspace directories.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}