Hallmark Testing Strategy: How 58 Automated Gates Ensure Design Quality

The Hallmark testing strategy relies on a deterministic "slop-test" that validates every generated page against 58 gates covering layout, responsiveness, token discipline, and anti-patterns before any output is emitted.

The Nutlope/hallmark repository implements a self-contained quality assurance system that treats design constraints as executable code. Rather than relying on external testing frameworks, it codifies visual and functional requirements into a declarative checklist enforced at build time.

The Slop-Test: Core of the Testing Strategy

The slop-test is Hallmark's primary testing mechanism. It runs 58 validation gates on every generated page or component, split into two categories:

• Universal gates: Always executed regardless of theme
• Genre-specific gates: Applied only for certain design themes

The complete gate specification lives in [skills/hallmark/references/slop-test.md](https://github.com/Nutlope/hallmark/blob/main/skills/hallmark/references/slop-test.md), while the high-level integration is documented in [skills/hallmark/SKILL.md](https://github.com/Nutlope/hallmark/blob/main/skills/hallmark/SKILL.md) (lines 46-48).

The Testing Pipeline: Two-Phase Validation

Hallmark's testing strategy employs a pre-emit self-critique followed by the full slop-test execution.

Phase 1: Pre-Emit Self-Critique

Before any slop-test gates run, the system scores drafts on six design axes:

• Philosophy
• Hierarchy
• Execution
• Specificity
• Restraint
• Variety

Any score below 3 triggers an automatic revision pass. This early filtering prevents low-quality drafts from advancing to expensive gate evaluation.

Phase 2: Gate Execution

The 58 gates validate across five major categories:

Category	Example Gates	Enforcement
Layout & spacing	Hero-fit (gate 44), sticky-nav (gate 56)	No horizontal scroll, proper viewport fit, correct padding ratios
Responsive behavior	Gates 34, 49	Clickable affordances never wrap across breakpoint boundaries
Token discipline	Gate 48	All color/font values must use var(--token-…) references
Anti-pattern detection	Gates 46-47, 38a	No fabricated metrics, no re-drawn chrome UI, no italic headings
Accessibility & motion	Gates 40-41	Contrast compliance, reduced-motion defaults

Gate Implementation Examples

The testing strategy manifests in concrete validation functions. Here's how three critical gates work in practice:

Hero-Fit Validation (Gate 44)

// Pseudo-code extracted from the test runner
function checkHeroFit(page) {
  const hero = page.querySelector('.hero');
  const viewportHeight = 800; // 13″ laptop reference
  const heroBottom = hero.getBoundingClientRect().bottom;

  // Gate 44 – hero must be fully visible without scrolling
  if (heroBottom > viewportHeight) {
    throw new Error('slop-test gate 44 failed: hero overflows the fold');
  }
}

This ensures above-the-fold visibility on standard laptop viewports.

Token-Only Color Enforcement (Gate 48)

function verifyTokenUsage(page) {
  const badColors = [...page.querySelectorAll('[style*="rgb("], [style*="#"]')];
  if (badColors.length) {
    throw new Error('slop-test gate 48 failed: inline colours detected');
  }
}

Raw rgb() or hex values fail the build; only CSS custom properties pass.

Anti-Pattern: Italic Headings (Gate 38a)

function checkHeadingStyles(page) {
  const italicHeadings = [...page.querySelectorAll('h1, h2, h3, h4, h5, h6')]
    .filter(h => getComputedStyle(h).fontStyle === 'italic');
  
  if (italicHeadings.length) {
    throw new Error('slop-test gate 38a failed: italic headings detected');
  }
}

This enforces typographic discipline per Hallmark's design system rules.

Visual Regression Fixtures

The testing strategy includes reference pages under site/_tests/ that serve as canaries for regression detection. Each fixture contains:

• index.html: The generated page under test
• Accompanying CSS assets
• Markdown brief describing expected behavior

Example: [site/_tests/05-tracejam-saas/index.html](https://github.com/Nutlope/hallmark/blob/main/site/_tests/05-tracejam-saas/index.html)

The test runner evaluates every gate against these fixtures after code changes, ensuring modifications don't violate established constraints.

Running the Test Suite

Execute the full Hallmark testing strategy via npm:

# Install dependencies (if not already present)

npm install

# Execute the full slop-test suite on all reference pages

npm test

The npm test script (declared in package.json) invokes the internal harness that:

1. Recursively reads site/_tests/*/index.html
2. Computes layout metrics (hero height, padding ratios, etc.)
3. Checks each of the 58 gates defined in slop-test.md
4. Emits a pass/fail summary; any failure aborts the CI run

Diversification Tracking

After successful slop-test completion, the system records metadata in .hallmark/log.json:

• Selected navigation archetype
• Selected footer archetype

This guarantees diversification across consecutive builds — no two runs share identical nav/footer combinations, preventing visual monotony while maintaining gate compliance.

Key Files in the Testing Architecture

File	Role in the Hallmark testing strategy
[skills/hallmark/SKILL.md](https://github.com/Nutlope/hallmark/blob/main/skills/hallmark/SKILL.md)	High-level description of pre-emit critique and slop-test integration
[skills/hallmark/references/slop-test.md](https://github.com/Nutlope/hallmark/blob/main/skills/hallmark/references/slop-test.md)	Authoritative checklist of all 58 gates
site/_tests/	Reference HTML/CSS fixtures for visual regression
package.json (scripts section)	Declares the "test" script entry point
.hallmark/log.json	Runtime metadata for cross-build diversification
skills/hallmark/references/component-cookbook.md	Component archetypes validated against relevant gates

Summary

Hallmark's testing strategy differs fundamentally from traditional web development approaches:

• Constraint-as-code: Design rules are executable gates, not documentation
• Deterministic validation: Same inputs always produce same pass/fail results
• Build-time enforcement: Quality gates block emission, not just warn
• Self-contained: No external test frameworks; pure Node-based harness
• Visual regression ready: Reference fixtures prevent degradation

This architecture makes the testing strategy tightly coupled to design system enforcement — every generated artifact must satisfy all 58 gates before reaching users.

Frequently Asked Questions

What makes Hallmark's slop-test different from unit testing?

Traditional unit tests verify function behavior in isolation; the slop-test validates visual and structural properties of rendered output. It operates on computed styles and DOM measurements rather than function returns, making it effectively a visual regression and design-linting hybrid that runs without browser screenshots.

Can the 58 gates be customized or extended?

According to the source code structure, gates are defined declaratively in skills/hallmark/references/slop-test.md. While the repository doesn't expose a public plugin API, the gate specification is human-readable markdown, suggesting the test harness parses this file to derive validation rules. Adding gates would require modifying both this specification and the corresponding test runner implementation.

How does Hallmark prevent testing the same layout repeatedly?

The diversification logic in .hallmark/log.json tracks which navigation and footer archetypes were selected in previous runs. Gate 32 specifically enforces that consecutive builds don't repeat combinations, ensuring variety while maintaining that every variant still passes the full 58-gate slop-test validation.

Does the slop-test run in CI/CD pipelines?

Yes. The npm test script is designed for CI integration — any gate failure throws an error that aborts the build. The human-readable error messages surface specific gate violations (e.g., "slop-test gate 44 failed") enabling rapid debugging without local reproduction.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}