Plugin Generation Pipeline from Collection Manifests in HVE Core

The HVE Core repository automates the creation of Copilot CLI plugins through a PowerShell-based plugin generation pipeline that processes *.collection.yml manifests, filters artifacts by maturity channel, and outputs ready-to-publish plugin directories with metadata and documentation.

The microsoft/hve-core project maintains a fully automated system that transforms YAML collection manifests into distributable plugins for GitHub Copilot CLI. Understanding the plugin generation pipeline from collection manifests is essential for contributors who need to package agents, prompts, and skills for different release channels. This pipeline handles everything from manifest discovery to marketplace indexing using pure PowerShell modules designed for CI integration and local development.

Manifest Discovery and Channel Filtering

The pipeline initiates by locating and validating collection definitions before applying release-channel constraints.

1. Discover collection manifests — The Get-AllCollections function in scripts/collections/Modules/CollectionHelpers.psm1 recursively scans the collections/ directory for *.collection.yml files and returns structured manifest objects.

2. Filter by release channel — The Find-CollectionManifestsCore function in scripts/extension/Find-CollectionManifests.ps1 removes deprecated collections unconditionally and skips experimental collections when the target channel is Stable. This ensures only production-ready artifacts proceed to packaging.

3. Refresh the meta-collection — Before generating any plugins, Update-HveCoreAllCollection (tested in scripts/tests/collections/CollectionHelpers.Tests.ps1) updates the hve-core-all.collection.yml file to reflect the latest artifact states across the repository.

Artifact Resolution and Maturity Filtering

Once manifests are validated, the pipeline resolves actual artifact files and enforces maturity constraints.

4. Resolve artifact sources — Get-ArtifactFiles (also in CollectionHelpers.psm1) walks the repository structure under .github/agents, .github/prompts, .github/instructions, and .github/skills, building a flat list of all artifact files tagged by kind (agent, prompt, instruction, skill).

5. Apply channel-specific maturity — For each collection, Select-CollectionItemsByChannel in scripts/plugins/Generate-Plugins.ps1 drops items whose maturity field does not match the requested channel (e.g., excluding experimental items from Stable builds).

Plugin Directory Construction

With filtered artifacts identified, the pipeline constructs the physical plugin structure and metadata files.

6. Write the plugin directory — Write-PluginDirectory in scripts/plugins/Modules/PluginHelpers.psm1 creates the folder hierarchy at plugins/<collection-id>/ with sub-folders for agents, commands, instructions, and skills. For each artifact, it creates symbolic links when the host supports them or text stubs that Git can rewrite as symlinks later. It also extracts front-matter descriptions for documentation.

7. Emit plugin.json — Using New-PluginManifestContent (same module), the pipeline generates a JSON manifest containing the plugin name, description, and repository version sourced from package.json.

8. Emit README.md — New-PluginReadmeContent builds a markdown README with installation instructions, a table of every included artifact, and optional maturity notices for experimental or preview channels.

Marketplace Aggregation and Orchestration

The final stages aggregate individual plugins into a searchable marketplace index and coordinate the entire workflow.

9. Write the marketplace index — After all plugins are generated, Write-MarketplaceManifest (in PluginHelpers.psm1) creates a global marketplace.json under .github/plugin/. This file aggregates every plugin's manifest and supplies repository-level metadata required by the Copilot Marketplace.

10. Orchestrate the whole flow — The top-level scripts/plugins/Generate-Plugins.ps1 wires all functions together. It discovers collections, filters them by channel, generates each plugin directory, and emits the marketplace file. It supports -DryRun for validation, -Refresh to force meta-collection updates, and explicit -CollectionIds or -Channel arguments for targeted builds.

Key Implementation Files

File	Role
scripts/extension/Find-CollectionManifests.ps1	Generates the CI matrix by filtering collection manifests according to channel.
scripts/plugins/Generate-Plugins.ps1	Top-level orchestrator that drives the entire plugin generation process.
scripts/collections/Modules/CollectionHelpers.psm1	Core library for loading manifests, discovering artifacts, and handling maturity states.
scripts/plugins/Modules/PluginHelpers.psm1	Functions that create plugin directories, manage symlinks, write plugin.json, generate README.md, and build the marketplace index.
scripts/tests/plugins/Generate-Plugins.Tests.ps1	Test suite validating end-to-end pipeline behavior.
scripts/tests/collections/CollectionHelpers.Tests.ps1	Unit tests for collection discovery, maturity filtering, and meta-collection updates.

Running the Pipeline Locally

Generate every plugin for the stable channel using default settings:

# From the repository root

.\scripts\plugins\Generate-Plugins.ps1

Generate only specific collections in dry-run mode to preview changes without writing files:

.\scripts\plugins\Generate-Plugins.ps1 `
    -CollectionIds rpi,github `
    -DryRun `
    -Channel Stable

Inspect the CI matrix used for collection discovery across different channels:

.\scripts\extension\Find-CollectionManifests.ps1 -Channel PreRelease

Programmatically generate a single plugin from another script:

# Load helper modules

Import-Module .\scripts\plugins\Modules\PluginHelpers.psm1 -Force
Import-Module .\scripts\collections\Modules\CollectionHelpers.psm1 -Force

# Resolve collections

$all = Get-AllCollections -CollectionsDir "$PSScriptRoot\collections"

# Filter for stable channel

$stable = Select-CollectionItemsByChannel -Collection $all[0] -Channel 'Stable'

# Write a single plugin (dry-run)

Write-PluginDirectory `
    -Collection $stable `
    -PluginsDir "$PSScriptRoot\plugins" `
    -RepoRoot $PSScriptRoot `
    -Version '0.1.0' `
    -DryRun

Summary

• The plugin generation pipeline from collection manifests is implemented entirely in PowerShell within the microsoft/hve-core repository.
• Collection discovery relies on Get-AllCollections and Find-CollectionManifestsCore to locate and filter *.collection.yml files by maturity channel.
• Artifact resolution scans .github/ subdirectories to map agents, prompts, instructions, and skills to their parent collections.
• Channel filtering occurs at two stages: initial manifest filtering removes deprecated collections, and Select-CollectionItemsByChannel removes individual artifacts that don't match the target stability level.
• Plugin construction creates directory structures with symlinks or stubs, generates plugin.json metadata, and produces README.md documentation.
• Marketplace aggregation produces a global marketplace.json index consumed by the Copilot Marketplace.
• All components are tested in scripts/tests/ and support both CI automation and local development workflows via parameters like -DryRun and -CollectionIds.

Frequently Asked Questions

What is a collection manifest in HVE Core?

A collection manifest is a YAML file (conventionally named *.collection.yml) that declaratively groups Copilot CLI artifacts—such as agents, prompts, instructions, and skills—into a cohesive plugin. Each manifest specifies the artifact paths, maturity levels (stable, preview, experimental), and metadata required by the plugin generation pipeline to construct distributable packages.

How does the pipeline distinguish between stable and experimental releases?

The pipeline applies channel filtering at two distinct points. First, Find-CollectionManifestsCore excludes deprecated collections entirely and filters out experimental collections when the -Channel Stable parameter is passed. Second, for each retained collection, Select-CollectionItemsByChannel evaluates the maturity field of individual artifacts, ensuring only artifacts matching the requested channel (Stable, PreRelease, or Experimental) are included in the final plugin directory and plugin.json manifest.

Can I generate plugins for specific collections instead of the entire repository?

Yes. The top-level Generate-Plugins.ps1 accepts an array of collection identifiers via the -CollectionIds parameter. When provided, the pipeline limits discovery and generation to only those specified collections, significantly reducing build times during development or targeted updates. Combine this with -DryRun to validate the output without modifying the filesystem.

What files constitute a generated plugin package?

Each generated plugin consists of a directory named after the collection ID containing symbolic links (or stub files) to artifacts organized into agents/, commands/, instructions/, and skills/ subdirectories, accompanied by a plugin.json manifest file and a README.md documentation file. After all plugins are generated, the pipeline additionally creates a marketplace.json file in .github/plugin/ that indexes all available plugins for the Copilot Marketplace.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}