How Hallmark's Audit Verb Scores Code Against Anti-Patterns

Hallmark's audit verb evaluates codebases by parsing source files into ASTs, matching structural signatures against a markdown-defined anti-pattern catalogue, and normalizing detected violations to a 0-100 severity score.

The audit verb in Nutlope/hallmark is a specialized skill that quantifies code quality by detecting architectural anti-patterns across your repository. Unlike traditional linters that flag stylistic issues, Hallmark's implementation focuses on structural problems defined in a human-readable catalogue, assigning each violation a weighted severity that contributes to an overall audit score.

Architecture of the Hallmark Audit Verb

The audit verb operates through a three-stage pipeline implemented in skills/hallmark/SKILL.md. This design separates pattern definition from detection logic, allowing the tool to adapt to new anti-patterns without modifying the core engine.

Loading the Anti-Pattern Catalogue

At runtime, the audit verb reads skills/hallmark/references/anti-patterns.md, a markdown file that serves as the single source of truth for all detectable issues. Each entry defines a pattern signature, description, and severity weight. The parser converts this document into an in-memory map used for matching against source code ASTs.

AST Parsing and Pattern Matching

When you invoke hallmark audit <path>, the verb recursively collects source files and parses them into Abstract Syntax Trees using language-specific parsers (e.g., @babel/parser for JavaScript/TypeScript as specified in package.json). The engine traverses these trees, comparing node structures against the catalogue signatures. Each match records the file path, line number, and associated weight.

Score Calculation and Normalization

After scanning, Hallmark aggregates the total weight of all findings and normalizes it against the maximum possible score from the catalogue. The engine applies the formula Math.round((totalWeight / maxPossible) * 100) to produce a final score between 0 (no anti-patterns detected) and 100 (codebase saturated with high-severity issues).

Core Implementation in SKILL.md

The registration and orchestration logic resides in skills/hallmark/SKILL.md, where the verb handler coordinates the audit process:

// Excerpt from skills/hallmark/SKILL.md
skill.registerVerb('audit', async (args) => {
  const catalogue = await loadAntiPatternCatalogue(); // reads anti-patterns.md
  const files = await collectSourceFiles(args.path);
  const findings = [];

  for (const file of files) {
    const ast = parseToAST(file.content);
    for (const pattern of catalogue) {
      if (matchesPattern(ast, pattern.signature)) {
        findings.push({
          file: file.path,
          line: locateLine(ast, pattern.signature),
          weight: pattern.weight,
          description: pattern.description,
        });
      }
    }
  }

  const totalWeight = findings.reduce((s, f) => s + f.weight, 0);
  const maxPossible = catalogue.reduce((s, p) => s + p.weight, 0);
  const score = Math.round((totalWeight / maxPossible) * 100);

  reportAuditResult(score, findings);
});

This implementation demonstrates how Hallmark decouples pattern definitions from detection algorithms, enabling the audit verb to support multiple languages through pluggable AST parsers.

Running a Hallmark Audit

Execute the audit from your project root to receive a quantitative assessment of anti-pattern density:

hallmark audit ./src

Typical output follows this structured format:

Audit Score: 27 / 100

Detected anti-patterns:
  • src/utils/helpers.js:12 – "Deeply nested callbacks" (weight 5)
  • src/components/Widget.tsx:45 – "Excessive prop drilling" (weight 8)
  • src/api/client.ts:78 – "Hard-coded URLs" (weight 4)

Interpreting Your Audit Score

A score of 0 indicates a clean codebase with no catalogue violations, while higher scores reflect accumulated technical debt weighted by severity. Individual file paths and line numbers allow developers to prioritize fixes based on the specific anti-patterns detected and their contribution to the total score.

Extending the Anti-Pattern Catalogue

To customize scoring criteria, edit skills/hallmark/references/anti-patterns.md and append new entries:

### Unused imports

- **Description:** Files contain imports that are never referenced.
- **Signature:** `ImportDeclaration` nodes with no Identifier usages.
- **Weight:** 3

The next audit run automatically incorporates these definitions without requiring changes to the core verb implementation in SKILL.md.

Summary

• Hallmark's audit verb is registered in skills/hallmark/SKILL.md and evaluates code against architectural anti-patterns rather than stylistic issues.
• The anti-pattern catalogue lives in skills/hallmark/references/anti-patterns.md and defines detection signatures with severity weights.
• Source code is parsed into ASTs and traversed to match against catalogue entries, recording file paths and line numbers for each violation.
• The final 0-100 score normalizes total violation weight against the catalogue maximum, with 0 representing perfect code and 100 representing maximum anti-pattern saturation.
• Users can extend detection capabilities by adding markdown entries to the catalogue without modifying the audit verb's source code.

Frequently Asked Questions

What file does Hallmark use to define anti-patterns?

Hallmark stores its anti-pattern definitions in skills/hallmark/references/anti-patterns.md. This markdown file contains structured entries with descriptions, AST signatures, and severity weights that the audit verb loads at runtime before scanning begins.

How is the Hallmark audit score calculated?

The score aggregates the weight of all detected anti-patterns and applies the formula (totalWeight / maxPossible) * 100. This produces a normalized integer between 0 and 100, where 0 indicates no anti-patterns found and 100 represents a codebase matching every high-severity pattern in the catalogue.

Can I add custom anti-patterns to Hallmark's audit?

Yes. Add new entries to skills/hallmark/references/anti-patterns.md with a markdown header, description, signature definition, and weight value. The audit verb automatically reads this file on each execution, so custom patterns are immediately available without code changes or redeployment.

What programming languages does the Hallmark audit verb support?

The audit verb supports any language for which Hallmark bundles a parser, typically JavaScript and TypeScript via @babel/parser as referenced in the repository's package.json. The AST generation step uses these language-specific parsers to create traversable trees for pattern matching against the catalogue signatures.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}