How Hallmark Handles Testing: A Design-First Approach to Visual Validation

Hallmark replaces traditional unit testing with a 58‑gate "slop‑test" checklist and visual regression fixtures stored under site/_tests to validate every generated page.

The Nutlope/hallmark repository is an AI‑powered design system that generates production‑ready HTML/CSS pages. Unlike conventional JavaScript projects, how testing is handled in the Hallmark repository centers on deterministic design validation rather than code coverage metrics. The system enforces visual consistency, accessibility compliance, and token discipline through domain‑specific quality gates.

The Slop‑Test: A 58‑Gate Design Checklist

The slop‑test is Hallmark's primary quality mechanism. It lives in skills/hallmark/references/slop-test.md and runs before any artifact is emitted.

How the Slop‑Test Works

1. Pre‑emit critique – The core skill (skills/hallmark/SKILL.md) scores the artifact across six axes.
2. Gate validation – All 58 gates must pass; failures trigger automatic revision or abort.

Key gates include:

• Responsive affordances – No button label may wrap at any viewport (gate 49)
• WCAG contrast – Every foreground/background pair must meet accessibility thresholds (gates 40‑41)
• Motion safety – Required prefers-reduced-motion overrides (gate 44)
• Token discipline – All colors and fonts must reference design tokens; inline hex or rgb() values fail (gate 48)
• Anti‑pattern bans – No fake browser chrome, no invented metrics, no italic headings (gates 46‑47, 38a)

Pre‑Emit Critique Implementation

This pseudo‑code from SKILL.md illustrates the validation flow:

// pre-emit self-critique
function preEmitCritique(artifact) {
  const scores = scoreSixAxes(artifact);
  if (scores.some(s => s < 3)) {
    // trigger a revision pass
    return revise(artifact);
  }
  // run the 58-gate slop test
  const slopResults = runSlopTest(artifact);
  if (!slopResults.every(g => g.pass)) {
    throw new Error('Slop test failed – aborting emit');
  }
  return artifact;
}

Visual Regression Tests in site/_tests/

The visual regression test suite provides concrete fixtures that exercise every macro‑structure, theme, and component. These are not automated screenshot comparisons—they are curated reference implementations that demonstrate expected output quality.

Test Fixture Structure

Each folder under site/_tests/ contains:

File	Purpose
index.html	Full page markup output
style.css	Theme‑specific CSS
brief.md	Design brief fed to the skill
notes.md	Human‑readable expected behavior

Example Test Fixtures

• site/_tests/01-tide-podcast/ – SaaS‑style podcast landing page
• site/_tests/03-maple-bakery/ – Bakery brand hero with CSS‑SVG art
• site/_tests/05-tracejam-saas/ – Observability product page with tilted CSS art

The 05-tracejam-saas brief demonstrates the specificity expected:

# Brief

A SaaS observability product.  
Hero: left-aligned headline "Distributed tracing that explains itself."  
Right side: hand-crafted CSS-art trace waterfall, tilted -0.4°, extending 12vw beyond the viewport.

Generated output passes through the same slop‑test gates before return.

Verb‑Specific Testing

The site/_tests/verbs/ directory validates Hallmark's command verbs:

• audit – Analyzes existing pages against the slop‑test
• study – Extracts design DNA from reference URLs
• redesign – Transforms input pages while preserving compliance

Each verb test provides input HTML/CSS and expected output that must reproduce the transformation while passing all 58 gates.

Component Cookbook for Deterministic Rendering

The component cookbook (skills/hallmark/references/component-cookbook.md) indexes ~50 component archetypes (e.g., H1, F3, N5). During builds:

1. The skill identifies required components from the macro‑structure
2. Pulls only relevant specs from the cookbook
3. Generates minimal, deterministic output

This tree‑shaking approach ensures consistent rendering across all test fixtures.

Key Files in the Testing Architecture

File Path	Role in Testing
skills/hallmark/references/slop-test.md	Authoritative 58‑gate checklist
skills/hallmark/SKILL.md	Core driver invoking validation
skills/hallmark/references/component-cookbook.md	Component archetype specifications
skills/hallmark/references/anti-patterns.md	"Anti‑slop" rule definitions
site/_tests/	Visual regression fixtures per theme/structure

Summary

• No traditional unit tests – Hallmark uses design‑centric validation instead of Jest or Mocha
• 58‑gate slop‑test enforces visual, accessibility, and token discipline on every output
• site/_tests/ fixtures provide concrete regression targets for themes and components
• Verb‑specific tests validate audit, study, and redesign workflows
• Component cookbook enables deterministic, minimal rendering

Frequently Asked Questions

Does Hallmark use Jest, Mocha, or other JavaScript testing frameworks?

No. According to the Hallmark source code, the repository deliberately avoids conventional unit‑testing frameworks. Validation relies entirely on the slop‑test checklist and visual regression fixtures in site/_tests/.

What happens if a generated page fails the slop‑test?

The skill aborts emit and triggers automatic revision. The preEmitCritique function in SKILL.md scores the artifact; any failing gate throws an error with specific feedback for correction.

How are visual regression tests maintained without screenshot comparison?

Test fixtures are curated HTML/CSS implementations with human‑reviewable notes.md files. They serve as qualitative benchmarks rather than pixel‑perfect baselines, allowing designer judgment alongside automated gate validation.

Can the slop‑test gates be customized or extended?

Yes. The skills/hallmark/references/slop-test.md file is the authoritative source. Modifying this checklist updates validation for all subsequent builds, as the core skill reads gates directly from this reference file.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}