How Hallmark's Pre-emit Self-Critique Works: Automated Quality Gates Before Output

Hallmark's Pre-emit Self-Critique is a deterministic quality-check that scores every generated artifact on six axes—Philosophy, Hierarchy, Execution, Specificity, Restraint, and Variety—from 1 to 5, stamps the results as a standardized comment at the file's top, and automatically regenerates the output if any score falls below 3.

The Nutlope/hallmark repository implements this rigorous evaluation mechanism to ensure all generated code adheres to strict design standards before reaching the user. By embedding quality validation directly into the generation pipeline, Hallmark prevents low-quality outputs from ever being emitted.

The Two-Step Quality Process

According to the Hallmark skill definition in skills/hallmark/SKILL.md at line 46, the Pre-emit Self-Critique operates through two deterministic phases:

1. Score the output on six axes – The system evaluates Philosophy, Hierarchy, Execution, Specificity, Restraint, and Variety, assigning each a rating from 1 to 5. This scoring occurs after content generation but before the output is returned to the caller.

2. Stamp the scores onto the artifact – The six scores are written as a comment at the very top of the file using the exact format specified in skills/hallmark/references/slop-test.md.

This ensures every artifact carries verifiable quality metadata before emission.

The Six-Axis Evaluation Framework

Hallmark evaluates every output across six distinct quality dimensions. Each axis receives a numeric rating from 1 to 5, where 5 represents optimal adherence to Hallmark's architectural principles.

The evaluation axes are:

• Philosophy (P): Alignment with core design principles and conceptual coherence
• Hierarchy (H): Structural organization and visual priority management
• Execution (E): Technical implementation quality and correctness
• Specificity (S): Precision of selectors and property definitions
• Restraint (R): Appropriate limitation of complexity and scope creep
• Variety (V): Balanced differentiation without visual chaos

Comment Stamping Format and Syntax

After scoring, the system embeds results as a machine-readable comment at the very beginning of the generated file. According to the reference implementation in skills/hallmark/references/slop-test.md at line 24, the format follows this exact pattern:

/* Hallmark · pre‑emit critique: P5 H4 E5 S4 R5 V5 */

The comment structure is rigid: /* Hallmark · pre‑emit critique: Px Hx Ex Sx Rx Vx */, where each letter-number pair corresponds to the respective axis score. This stamp appears immediately at the top of CSS files, HTML documents, and other generated artifacts.

Automatic Revision Thresholds

The system enforces quality through a hard threshold mechanism: if any axis scores below 3, Hallmark triggers an automatic revision pass. The generation loop re-executes, producing new content that undergoes the same six-axis evaluation. This cycle continues recursively until all scores meet the minimum threshold, ensuring no substandard artifacts reach the caller.

As implemented in the generation pipeline, this creates a self-correcting loop where the model critiques its own output before emission, effectively acting as a deterministic quality gate.

Implementation in the Source Code

The complete specification for this mechanism resides in skills/hallmark/SKILL.md, which defines the Pre-emit self-critique as a mandatory step in the Hallmark workflow. The document specifies that scoring occurs after content generation but before returning output to the caller.

Real-world implementations appear throughout the repository. For example, site/examples/wayfare/style.css contains generated stylesheets with the critique comment as the first line. Test pages such as site/_tests/13-alma/index.html demonstrate how the metadata appears in HTML outputs.

Programmatic Workflow Example

The following pseudo-code illustrates how the Pre-emit Self-Critique integrates into the generation pipeline:

def pre_emit_critique(output):
    scores = {
        "P": evaluate_philosophy(output),
        "H": evaluate_hierarchy(output),
        "E": evaluate_execution(output),
        "S": evaluate_specificity(output),
        "R": evaluate_restraint(output),
        "V": evaluate_variety(output),
    }

    # Enforce minimum quality threshold of 3

    if any(v < 3 for v in scores.values()):
        output = regenerate_output()
        return pre_emit_critique(output)  # Recursive validation

    comment = "/* Hallmark · pre-emit critique: " + " ".join(
        f"{k}{v}" for k, v in scores.items()
    ) + " */\n"
    return comment + output

This implementation ensures that every artifact passing through the system carries verified quality metadata and meets the minimum standards defined in the Hallmark skill specification.

Summary

• Hallmark's Pre-emit Self-Critique evaluates every output on six axes (Philosophy, Hierarchy, Execution, Specificity, Restraint, Variety) rated 1-5 before emission
• Scores are stamped as standardized comments at the top of files using the format /* Hallmark · pre‑emit critique: Px Hx Ex Sx Rx Vx */
• Any score below 3 triggers an automatic regeneration loop until all quality thresholds are satisfied
• The mechanism is defined in skills/hallmark/SKILL.md at line 46 and demonstrated in skills/hallmark/references/slop-test.md at line 24
• Live examples appear in generated files under site/examples/ and test pages in site/_tests/

Frequently Asked Questions

What are the six evaluation axes in Hallmark's Pre-emit Self-Critique?

The six axes are Philosophy (P), Hierarchy (H), Execution (E), Specificity (S), Restraint (R), and Variety (V). Each axis measures a distinct quality dimension of the generated artifact, receiving a score from 1 (poor) to 5 (excellent) before the output is finalized and emitted to the user.

What happens if a generated artifact scores below 3 on any axis?

If any axis receives a rating below 3, Hallmark automatically triggers a revision pass. The generation loop re-runs, creating new content that undergoes the same six-axis evaluation. This process repeats until every axis achieves a minimum score of 3, ensuring only high-quality artifacts reach the caller.

Where is the Pre-emit Self-Critique defined in the Hallmark source code?

The primary definition appears in skills/hallmark/SKILL.md at line 46, which outlines the quality-check workflow as part of the Hallmark skill definition. The exact comment stamp format is documented in skills/hallmark/references/slop-test.md at line 24, with live examples visible in generated CSS files under site/examples/ and test files in site/_tests/.

How is the critique comment formatted in generated files?

The comment follows a strict syntax: /* Hallmark · pre‑emit critique: P5 H4 E5 S4 R5 V5 */, where each letter-number pair represents the score for Philosophy, Hierarchy, Execution, Specificity, Restraint, and Variety respectively. This comment always appears as the very first line of the generated artifact, immediately preceding the actual content.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}