How to Test a Specific Skill in Nutlope/hallmark

To test a specific skill in Nutlope/hallmark, set the SKILL environment variable to the target skill name and run npm run test:skill to execute isolated Playwright visual regression tests against that skill's fixtures.

The Nutlope/hallmark repository organizes content into modular "skills"—tiny, self-contained units such as audit or redesign stored under skills/hallmark/. Each skill includes dedicated visual regression tests located in site/_tests/verbs/ that verify HTML output, CSS styling, and Markdown rendering. Testing a specific skill allows you to validate changes quickly without executing the full test suite.

Locate the Skill Test Assets

According to the Nutlope/hallmark source code, every skill has a corresponding test folder under site/_tests/verbs/. For example, the audit skill (defined in skills/hallmark/SKILL.md) stores its test fixtures at:

• site/_tests/verbs/audit/input.html – The input fixture used by the test runner
• site/_tests/verbs/audit/notes.md – Supplementary notes for the test case
• site/_tests/verbs/audit/audit-report.md – The expected output snapshot

Replace audit with your target skill name to locate the relevant files for any other skill in the repository.

Run the Visual Regression Test for a Single Skill

The repository uses a Playwright-based visual-diff script defined in package.json. To test a specific skill, pass the skill name via the SKILL environment variable before invoking the npm script:

npm ci
SKILL=audit npm run test:skill

This command renders the skill's Markdown source, snapshots the generated HTML and CSS, and compares the output against the golden files in site/_tests/verbs/<skill>/. The harness reports PASS when the generated markup matches the snapshot exactly, or FAIL when discrepancies are detected.

Interpret Failures and Update Snapshots

When a test fails, the console prints a line-by-line diff indicating mismatched content. The system also generates visual diff images in site/_tests/_diffs/ to highlight rendering changes pixel-by-pixel.

If the changes are intentional, update the snapshot baseline by appending the --update-snapshots flag:

SKILL=audit npm run test:skill -- --update-snapshots

After updating, commit the modified files from site/_tests/verbs/<skill>/ along with any new reference images in _diffs/ to preserve the new expected behavior.

Key Files in the Testing Architecture

Understanding the test implementation requires familiarity with these specific files:

• package.json – Declares the test:skill npm script and lists Playwright as a dev dependency
• skills/hallmark/SKILL.md – Example skill source (Markdown) compiled during the test run
• site/_tests/verbs/<skill>/input.html – Input fixture consumed by the visual test runner
• site/_tests/verbs/<skill>/<skill>-report.md – Expected output snapshot used for comparison
• site/_tests/README.md – Documentation explaining the harness mechanics and fixture structure

Summary

• Skill isolation – Use the SKILL environment variable to target individual skills without executing npm run test:all
• Test location – Fixtures reside in site/_tests/verbs/<skill-name>/ with input.html and *-report.md files
• Execution – Run SKILL=<name> npm run test:skill to invoke Playwright visual regression tests
• Failure analysis – Check site/_tests/_diffs/ for visual comparisons when tests fail
• Baseline updates – Append -- --update-snapshots to record intentional rendering changes

Frequently Asked Questions

How do I determine which skills are available for testing?

Navigate to the skills/hallmark/ directory to view all skill definitions, or inspect the subdirectories under site/_tests/verbs/ to see which skills currently have test fixtures implemented. Each folder name in site/_tests/verbs/ corresponds directly to a valid skill name for the SKILL environment variable.

What testing framework does the repository use?

As implemented in Nutlope/hallmark, the test harness uses Playwright for visual regression testing. The test:skill script invokes the Playwright runner with the --skill flag internally, comparing rendered HTML output against stored snapshots to detect unintended layout or styling changes.

Can I test multiple specific skills in a single command?

The harness is designed for single-skill isolation via the SKILL environment variable. To test multiple specific skills, execute the command multiple times with different skill names, or run npm test to execute the complete suite against all skills simultaneously.

Where are the visual diff images stored when tests fail?

Failed comparisons generate visual diff images in site/_tests/_diffs/. These files provide a pixel-by-pixel comparison between the current rendering and the expected snapshot, allowing you to identify exactly which UI elements have changed.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}