# How to Test a Custom Skill Before Submission in the pm-skills Repository > Easily test your custom skill before submission to the phuryn/pm-skills repository. Use validation scripts and runtime checks to ensure a smooth submission process. - Repository: [Pawel Huryn/pm-skills](https://github.com/phuryn/pm-skills) - Tags: how-to-guide - Published: 2026-06-19 --- **Run `python validate_plugins.py` to validate metadata and schema, then execute `pm test-scenarios ` to verify runtime behavior against expected outputs before submitting to the marketplace.** Testing a custom skill before submission ensures your contribution meets the **pm-skills** repository standards and functions correctly in the PM-Toolkit ecosystem. The repository provides automated validation tools and scenario testing commands that verify both structural integrity and runtime behavior. Following the official testing workflow prevents broken submissions and guarantees your skill behaves as described under realistic conditions. ## Prerequisites and File Structure Before running tests, organize your skill files according to the repository layout and register the skill in the plugin manifest. ### Create the Skill Directory Place your skill under `pm-toolkit/skills//` containing a [`SKILL.md`](https://github.com/phuryn/pm-skills/blob/main/SKILL.md) file. This directory houses all files specific to your skill, including documentation and metadata. ### Register in the Plugin Manifest Add an entry to [`pm-toolkit/.claude-plugin/plugin.json`](https://github.com/phuryn/pm-skills/blob/main/pm-toolkit/.claude-plugin/plugin.json) specifying the skill name, type, and path to your [`SKILL.md`](https://github.com/phuryn/pm-skills/blob/main/SKILL.md): ```json { "name": "my-awesome-skill", "type": "skill", "path": "pm-toolkit/skills/my-awesome-skill/SKILL.md" } ``` The CLI uses this manifest to discover and load available skills. ## Step-by-Step Testing Workflow ### Step 1: Validate Plugin Metadata Run the repository validator to check syntactic correctness: ```bash python validate_plugins.py ``` The script, located at the repository root, parses every [`SKILL.md`](https://github.com/phuryn/pm-skills/blob/main/SKILL.md) and the plugin manifest, reporting missing fields or malformed JSON. A clean run indicates the skill passes the **syntactic** checks required for marketplace submission. ### Step 2: Create Scenario Tests Add a Markdown file under `pm-execution/skills/test-scenarios/.md` containing the test specification: ```markdown # Test: my-awesome-skill **Prompt** Write a 2-sentence brief for the idea: "AI-powered habit tracker". **Expected Output** - Must contain the phrase "habit tracker". - Must be ≤ 2 sentences. ``` This file defines the exact user input and the constraints the skill output must satisfy. ### Step 3: Execute Scenario Tests Run the scenario testing harness from the repository root: ```bash pm test-scenarios my-awesome-skill ``` As documented in [`pm-execution/commands/test-scenarios.md`](https://github.com/phuryn/pm-skills/blob/main/pm-execution/commands/test-scenarios.md), this command feeds the prompt to the skill, captures the response, and diff-compares it against the expected output constraints. ### Step 4: Manual CLI Verification After automated tests pass, invoke the skill directly to inspect output formatting and edge cases: ```bash pm run my-awesome-skill --prompt "Write a brief for: AI-powered habit tracker" ``` Test with empty inputs, long strings, and special characters to ensure robustness. ## Complete Code Examples ### Minimal SKILL.md Template Create [`pm-toolkit/skills/my-awesome-skill/SKILL.md`](https://github.com/phuryn/pm-skills/blob/main/pm-toolkit/skills/my-awesome-skill/SKILL.md): ```markdown # Skill: my-awesome-skill **Description** Generate a concise project brief from a short product idea. **Parameters** - `idea` (string): one-sentence product concept. **Example Prompt** ``` Write a 2-sentence brief for the idea: "AI-powered habit tracker". ``` **Result** A markdown paragraph summarising the product vision. ``` See a real example in the repository: [`pm-toolkit/skills/review-resume/SKILL.md`](https://github.com/phuryn/pm-skills/blob/main/pm-toolkit/skills/review-resume/SKILL.md). ### Updating the Plugin Manifest Edit [`pm-toolkit/.claude-plugin/plugin.json`](https://github.com/phuryn/pm-skills/blob/main/pm-toolkit/.claude-plugin/plugin.json) to include your skill: ```json { "plugins": [ { "name": "my-awesome-skill", "type": "skill", "path": "pm-toolkit/skills/my-awesome-skill/SKILL.md" } ] } ``` ### Running the Validator Execute from the repository root: ```bash python validate_plugins.py # → Output: 0 errors, 0 warnings ``` The validator checks that all registered skills have valid [`SKILL.md`](https://github.com/phuryn/pm-skills/blob/main/SKILL.md) files and that the JSON syntax in [`plugin.json`](https://github.com/phuryn/pm-skills/blob/main/plugin.json) is correct. ### Adding a Scenario Test Create [`pm-execution/skills/test-scenarios/my-awesome-skill.md`](https://github.com/phuryn/pm-skills/blob/main/pm-execution/skills/test-scenarios/my-awesome-skill.md): ```markdown # Test: my-awesome-skill **Prompt** Write a 2-sentence brief for the idea: "AI-powered habit tracker". **Expected Output** - Must contain the phrase "habit tracker". - Must be ≤ 2 sentences. ``` ### Executing the Test Run the specific scenario test: ```bash pm test-scenarios my-awesome-skill # → PASSED: output matches expected constraints ``` ### Manual CLI Invocation Perform final sanity checks: ```bash pm run my-awesome-skill --prompt "Write a brief for: AI-powered habit tracker" ``` ## Summary - **Validate first**: Run `python validate_plugins.py` to catch syntax errors and missing metadata before functional testing. - **Test scenarios**: Place scenario files in `pm-execution/skills/test-scenarios/` and run `pm test-scenarios ` to verify behavior against expected outputs. - **Register properly**: Ensure your skill is listed in [`pm-toolkit/.claude-plugin/plugin.json`](https://github.com/phuryn/pm-skills/blob/main/pm-toolkit/.claude-plugin/plugin.json) with the correct path to [`SKILL.md`](https://github.com/phuryn/pm-skills/blob/main/SKILL.md). - **Manual verification**: Use `pm run ` to test edge cases and formatting that automated tests might miss. - **Iterate**: Repeat the validation and testing cycle until all checks pass and the output meets quality standards. ## Frequently Asked Questions ### What does validate_plugins.py check exactly? The script validates that every skill registered in [`pm-toolkit/.claude-plugin/plugin.json`](https://github.com/phuryn/pm-skills/blob/main/pm-toolkit/.claude-plugin/plugin.json) has a corresponding [`SKILL.md`](https://github.com/phuryn/pm-skills/blob/main/SKILL.md) file with the required header structure, valid JSON syntax in the manifest, and all required CLI files present. It reports missing fields, malformed paths, and schema violations that would prevent the skill from loading in the PM-Toolkit. ### Where do I place scenario test files? Create Markdown files in `pm-execution/skills/test-scenarios/` named after your skill (e.g., [`my-awesome-skill.md`](https://github.com/phuryn/pm-skills/blob/main/my-awesome-skill.md)). These files should contain a **Prompt** section with the exact input and an **Expected Output** section defining the constraints or content the skill must produce. ### Can I test a skill without registering it in plugin.json? No. The `pm test-scenarios` command and `pm run` command both rely on the plugin registry to locate and load skill definitions. You must add an entry to [`pm-toolkit/.claude-plugin/plugin.json`](https://github.com/phuryn/pm-skills/blob/main/pm-toolkit/.claude-plugin/plugin.json) pointing to your [`SKILL.md`](https://github.com/phuryn/pm-skills/blob/main/SKILL.md) before the testing commands can recognize your skill. ### How do I debug a failing scenario test? Check the diff output from `pm test-scenarios` to see where the actual output diverged from the expected constraints. Verify that your [`SKILL.md`](https://github.com/phuryn/pm-skills/blob/main/SKILL.md) logic handles the specific prompt correctly, and ensure the expected output constraints in your scenario file are not overly restrictive. Run `pm run ` with the same prompt to inspect the raw output manually.