OpenAI Plugins `skills/` Directory Structure and `SKILL.md` Processing Explained

In the openai/plugins repository, every skill is a self-contained folder under plugins/<plugin-name>/skills/ containing a mandatory SKILL.md file that the plugin-eval toolchain validates for front-matter compliance, size constraints, and broken links.

The openai/plugins repository organizes capabilities into modular units called skills. Each skill's documentation and metadata reside in a SKILL.md file processed by the evaluation pipeline. Understanding this structure ensures your plugins pass continuous integration checks and integrate correctly with the OpenAI agent runtime.

Directory Layout of the skills/ Folder

Each plugin contains a top-level skills/ directory. Inside, every skill occupies its own subfolder that must contain a SKILL.md file at its root.

Standard Subdirectory Structure

A typical skill folder contains the following:

• SKILL.md – The primary markdown file describing the skill, its front-matter, and links to supplementary documentation.
• references/ – Optional markdown files with detailed documentation that SKILL.md can link to (e.g., details.md, api-usage.md).
• assets/ – Images, icons, or other static assets referenced from the markdown.
• agents/ – YAML files describing the OpenAI agent configuration for the skill (e.g., openai.yaml).
• scripts/ – Helper scripts implementing the skill’s runtime behavior (Python, Node.js, etc.).
• README.md (optional) – Human-readable overview of the skill folder.

Real-World Examples from the Repository

The repository contains concrete implementations demonstrating this layout:

• plugins/zoom/skills/video-sdk/web/SKILL.md – The web variant of the Zoom Video SDK skill.
• plugins/circleci/skills/config/SKILL.md – A CircleCI configuration skill.
• plugins/zotero/skills/zotero/SKILL.md – The Zotero integration skill.

All follow the convention of a single SKILL.md at the skill root with optional references/, assets/, agents/, and scripts/ sub-folders.

How SKILL.md Files Are Processed

The core evaluation logic resides in plugins/plugin-eval/src/evaluators/skill.js. The evaluateSkill(skillRoot) function orchestrates the validation pipeline through the following steps:

File Discovery and Content Loading

1. Locate the file – The function builds the path using path.join(skillRoot, "SKILL.md"). If the file is missing, the evaluator emits a "skill-file-missing" error.
2. Read the content – readText(skillPath) loads the markdown as a UTF-8 string.

Front-Matter Validation

3. Parse front-matter – parseFrontmatter(content) extracts the YAML block at the top of the file. Allowed keys are defined in ALLOWED_FRONTMATTER_KEYS (e.g., name, description, license, allowed-tools).
4. Validate mandatory fields:
   • name – Must exist and match the hyphen-case pattern /^[a-z0-9-]+$/.
   • description – Must exist, be ≤ 1024 characters, and contain a "use when" trigger phrase.

Content Quality Checks

5. Check file size – The markdown is split into lines. Files greater than 800 lines generate a "skill-too-large" error, while files between 501-800 lines trigger a "skill-large" warning. This encourages authors to keep SKILL.md concise and move detailed content into references/.
6. Count code fences – countCodeFences(markdown) counts triple-backtick pairs; warnings are emitted if the number of code blocks exceeds budget thresholds.
7. Detect relative links – findRelativeLinks(markdown) extracts all [text](path) links that are not absolute URLs. Each link is verified to point to an existing file inside the same skill directory using pathExists. Broken links generate a "relative-links-broken" check.
8. Report unexpected keys – Any front-matter key outside the allowed set creates a "frontmatter-extra-keys" warning.

Output Artifacts

The evaluator returns an array of Check objects containing errors, warnings, and metrics (such as line count or code-fence count) consumed by downstream CI dashboards and the Plugin Playground UI.

Core Evaluation Utilities

The validation pipeline relies on several supporting modules:

• plugins/plugin-eval/src/lib/files.js – Provides pathExists, readText, walkFiles, and relativePath utilities. walkFiles enumerates all files under a skill root (excluding SKILL.md) to validate reference files.
• plugins/plugin-eval/src/lib/frontmatter.js – A YAML parser returning { data, errors } used to enforce the front-matter schema.
• plugins/plugin-eval/src/core/target.js – Resolves a given path to a skill root and verifies whether the final component is SKILL.md.

These utilities make the evaluation logic reusable for any plugin following the skills/ → <skill-name>/SKILL.md convention.

Programmatically Evaluating Skills

You can invoke the evaluation logic directly to validate skills during development or in custom tooling.

Manually Evaluating a Single Skill

import { evaluateSkill } from "plugins/plugin-eval/src/evaluators/skill.js";

const skillRoot = "/path/to/repo/plugins/zoom/skills/video-sdk/web";
const result = await evaluateSkill(skillRoot);

console.log(result.checks);   // array of Check objects (errors, warnings, etc.)
console.log(result.metrics); // e.g. { lineCount: 312, codeFences: 5 }

Walking All Skills in a Plugin

import { walkFiles } from "plugins/plugin-eval/src/lib/files.js";
import path from "node:path";

async function allSkills(pluginRoot) {
  const skillsDir = path.join(pluginRoot, "skills");
  const skillFolders = await walkFiles(skillsDir, { directories: true });
  // Filter only immediate sub-folders that contain a SKILL.md
  return skillFolders.filter(async (dir) =>
    (await pathExists(path.join(dir, "SKILL.md")))
  );
}

Detecting Broken Relative Links

import { readText } from "plugins/plugin-eval/src/lib/files.js";
import { findRelativeLinks } from "plugins/plugin-eval/src/evaluators/skill.js";

async function checkLinks(skillPath) {
  const md = await readText(path.join(skillPath, "SKILL.md"));
  const links = findRelativeLinks(md);
  for (const link of links) {
    const exists = await pathExists(path.join(skillPath, link));
    if (!exists) console.warn(`Broken link → ${link}`);
  }
}

Summary

• Every skill is a self-contained folder under plugins/<plugin-name>/skills/<skill-name>/ with a mandatory SKILL.md file at its root.
• The plugin-eval package validates skills via plugins/plugin-eval/src/evaluators/skill.js, enforcing front-matter schemas (including hyphen-case name fields and description length limits), file size constraints (>800 lines errors, 501-800 warnings), and link integrity.
• Relative links in SKILL.md are verified against local references/ and assets/ directories to ensure documentation completeness.
• Validation results are returned as structured Check objects suitable for CI pipelines and UI integrations.

Frequently Asked Questions

What happens if a SKILL.md file is missing from a skill directory?

The evaluateSkill() function in plugins/plugin-eval/src/evaluators/skill.js emits a "skill-file-missing" error when it cannot locate SKILL.md at the skill root, preventing the plugin from passing quality gates.

What are the file size limits for SKILL.md?

The validator splits the markdown into lines and enforces strict limits: files exceeding 800 lines generate a "skill-too-large" error, while files between 501-800 lines trigger a "skill-large" warning. This design encourages authors to move extended documentation into the references/ subfolder.

How does the validator handle relative links in SKILL.md?

The findRelativeLinks(markdown) function extracts all relative [text](path) references and verifies they point to existing files within the skill directory using pathExists. Broken links generate "relative-links-broken" checks.

Which front-matter keys are allowed in a SKILL.md file?

The evaluator checks against ALLOWED_FRONTMATTER_KEYS, which includes name, description, license, and allowed-tools. Any unexpected keys trigger a "frontmatter-extra-keys" warning, ensuring consistent metadata across the repository.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}