How to Create New Agent Skills Following the agentskills.io Standard

To create a new agent skill following the agentskills.io standard, define a SKILL.md file with YAML front-matter inside a plugin's skills/ directory, add an eval.yaml test scenario under tests/, and register the skill in the marketplace manifest at .agents/plugins/marketplace.json.

The dotnet/skills repository provides a framework for building task-focused AI agent capabilities using the open agentskills.io specification. Creating new agent skills requires strict adherence to the repository's plugin-based architecture, including mandatory metadata schemas and automated validation tests.

Repository Architecture Overview

The dotnet/skills repository organizes capabilities into plugins, each containing skills (task-focused playbooks) and optional agents (role definitions). A skill must reside in a specific directory structure to be discoverable by agent systems like Copilot, Claude, Cursor, and Codex.

The required layout follows this pattern:

plugins/
  <plugin>/
    plugin.json                ← plugin metadata
    skills/
      <skill-name>/
        SKILL.md               ← skill definition (required)
        scripts/…              ← optional helper scripts
        references/…           ← optional external docs
        assets/…               ← optional static assets
    agents/
      <agent-name>.agent.md   ← optional agent definition
tests/
  <plugin>/
    <skill-name>/
      eval.yaml                ← test scenarios for the skill

See the official layout conventions in the repository's CONTRIBUTING.md.

Step 1: Choose or Create a Plugin

Skills are grouped by domain. If an existing plugin matches your skill's focus (e.g., dotnet-test for testing scenarios), add the skill under its skills/ folder. Otherwise, create a new plugin:

1. Add plugins/<plugin-name>/plugin.json with the plugin metadata.
2. Update marketplace manifests in .agents/plugins/marketplace.json, .claude-plugin/marketplace.json, etc., to include the new plugin.
3. Add a CODEOWNERS entry in .github/CODEOWNERS for the new plugin and its corresponding tests.

Reference the plugin creation process in CONTRIBUTING.md for detailed requirements.

Step 2: Define the Skill Metadata

Create the skill folder at plugins/<plugin>/skills/<skill-name>/ and add a SKILL.md file. The file must begin with YAML front-matter containing at minimum the name and description fields:

---
name: <skill-name>
description: |
  <brief description of what the skill does, when to use it, and when *not* to use it>
---

Following the front-matter, include these recommended sections:

• Purpose – One-sentence outcome statement.
• When to use / When not to use – Guidance for the agent; repeat critical constraints from the front-matter.
• Inputs – Required files, commands, or environment variables.
• Workflow – Numbered steps with checkpoints (the core playbook).
• Validation – Success criteria (tests, linters, manual checks).
• Common pitfalls – Known traps and mitigations.

Review the thread-abort-migration skill in plugins/dotnet-upgrade/skills/thread-abort-migration/SKILL.md for a complete implementation example.

Step 3: Add Automated Validation Tests

Every skill requires an eval.yaml file under tests/<plugin>/<skill-name>/ defining one or more test scenarios:

scenarios:
  - name: "Identify Thread.Abort usage"
    prompt: "Migrate Thread.Abort calls in MyApp"
    assertions:
      - type: output_contains
        value: "CancellationTokenSource"
    rubric:
      - "Agent correctly identifies Thread.Abort patterns"
      - "Agent suggests cooperative-cancellation replacement"
    timeout: 120

The schema supports various assertion types and rubric criteria for grading agent responses. Consult eng/skill-validator/src/README.md for the complete assertion schema and validation rules.

Step 4: Register in Marketplace Manifests

Add the skill to .agents/plugins/marketplace.json so agents can discover it:

{
  "plugins": [
    {
      "name": "<plugin>",
      "source": "git+https://github.com/dotnet/skills.git",
      "skills": ["<skill-name>"]
    }
  ]
}

This registration enables discovery across supported agent platforms including Copilot, Claude, Cursor, and Codex.

Step 5: Update Documentation

Add the new plugin or skill to the "What's Included" table in the root README.md to ensure visibility on the repository dashboard:

| [dotnet-upgrade](plugins/dotnet-upgrade/) | Skills for migrating .NET projects… |

Step 6: Validate Locally Before Submitting

Run the skill-validator locally to catch errors before opening a pull request:

dotnet run --project eng/skill-validator/src/SkillValidator.csproj \
  -- evaluate \
  --tests-dir tests/<plugin> \
  plugins/<plugin>/skills/<skill-name>

Fix any validation failures. The CI workflow will automatically execute the skill-validator against your pull request; all tests must pass before merging.

Complete Example: Creating a "configure-jwt-auth" Skill

File structure for a new ASP.NET Core security skill:

plugins/dotnet-aspnetcore/
├─ plugin.json
├─ skills/
│  └─ configure-jwt-auth/
│     ├─ SKILL.md
│     ├─ scripts/
│     │   └─ generate-jwks.ps1
│     └─ references/
│         └─ jwt-spec.md
└─ agents/
   └─ api-security.agent.md

SKILL.md excerpt:

---
name: configure-jwt-auth
description: |
  Adds JWT authentication to an ASP.NET Core API project.
  USE FOR: securing new APIs, migrating from cookie auth.
  DO NOT USE FOR: projects that already use JWT or require custom token validation.
---

# Configure JWT Authentication

## Purpose

Enable JWT-based authentication for a new or existing ASP.NET Core API.

## Inputs

| Input | Required | Description |
|-------|----------|-------------|
| Project file | Yes | Path to the .csproj that defines the API |
| Issuer URL | Yes | URL of the token issuer |
| Audience | Yes | Expected audience claim |

## Workflow

1. Add `Microsoft.AspNetCore.Authentication.JwtBearer` NuGet package.
2. Insert the `AddJwtBearer` call in `Startup.ConfigureServices`.
3. Add `UseAuthentication` and `UseAuthorization` in the request pipeline.
4. Create a `JwtSettings` POCO and bind it from `appsettings.json`.
5. Verify the configuration with `dotnet run` and a test token.

## Validation

- `dotnet build` succeeds with no warnings.
- Running `curl <api>/protected` returns `401` before a token and `200` after a valid token.

## Common pitfalls

- Forgetting to call `app.UseAuthentication()` results in a `401` on every request.
- Not setting `ValidateIssuerSigningKey` leads to insecure token acceptance.

eval.yaml excerpt:

scenarios:
  - name: "Setup JWT auth"
    prompt: "Configure JWT authentication for MyApi"
    assertions:
      - type: output_contains
        value: "AddJwtBearer"
    timeout: 180

Summary

• Skills live in plugins: Create or select a plugin in plugins/, then add your skill to the skills/ subdirectory.
• SKILL.md is mandatory: Must include YAML front-matter with name and description, followed by structured sections.
• Testing is required: Every skill needs an eval.yaml in tests/<plugin>/<skill-name>/ defining scenarios and assertions.
• Registration required: Update .agents/plugins/marketplace.json and the root README.md for discoverability.
• Validate locally: Use eng/skill-validator/src/SkillValidator.csproj to test before submitting a PR.

Frequently Asked Questions

What is the agentskills.io standard?

The agentskills.io standard is an open specification for defining AI agent capabilities through structured markdown files with YAML front-matter, automated validation tests, and marketplace manifests. It ensures skills are discoverable, testable, and portable across different agent platforms like Copilot, Claude, and Cursor.

What happens if I omit the YAML front-matter in SKILL.md?

The skill-validator will reject the skill. The SKILL.md file must begin with YAML front-matter containing at minimum the name and description fields. Without this metadata, the skill cannot be indexed by the marketplace manifests or parsed by agent systems.

How do I test my skill before submitting a pull request?

Run the local validator using dotnet run --project eng/skill-validator/src/SkillValidator.csproj -- evaluate --tests-dir tests/<plugin> plugins/<plugin>/skills/<skill-name>. This executes your eval.yaml scenarios against the skill definition and reports any assertion failures or schema violations.

Can I include helper scripts with my skill?

Yes. Place optional helper scripts in the scripts/ subdirectory within your skill folder (e.g., plugins/<plugin>/skills/<skill-name>/scripts/). You can also include static assets in assets/ and external documentation references in references/. These are not required but support complex automation workflows.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}