Intended-vs-Implemented Skill: How to Audit Code Against Documentation in phuryn/pm-skills

The intended-vs-implemented skill is a systematic method that compares documented intent against actual source code enforcement to surface security gaps and compliance violations that traditional linters miss.

The intended-vs-implemented skill, defined in the phuryn/pm-skills repository, provides a structured framework for identifying where a system's documented rules diverge from its implementation. By treating markdown documentation as the source of truth and mapping it to concrete enforcement points in the codebase, this audit method reveals high-impact mismatches that automated tools typically overlook. It is particularly effective for validating AI-generated code, security policies, and architectural constraints.

What Is the Intended-vs-Implemented Skill?

The intended-vs-implemented skill defines a five-step method for discovering gaps between documented intent (permissions, architecture, data-flow rules) and actual implementation in source code. Unlike generic linters that lack semantic understanding of business rules, this skill requires the auditor to establish explicit claims from documentation and verify their enforcement through file-and-line citations. According to the skill specification in pm-ai-shipping/skills/intended-vs-implemented/SKILL.md, the process focuses on "crossing trust, cost, data, or tenant boundaries" to filter out cosmetic drift and highlight only security-critical discrepancies.

How the Audit Works

The audit follows a rigorous sequence designed to produce traceable evidence for every finding. Each step requires specific artifacts and yields concrete citations.

Step 1: Establish Intent

The auditor reads all markdown files under a designated /documentation/ folder (e.g., permissions.md, architecture.md). These files become the source of truth for what the system should enforce. As specified in the skill documentation, the process begins by reading the "/documentation/*.md set as the source of truth for what should be true."

Step 2: Gather Implementation Evidence

The codebase is scanned for concrete checks that enforce each documented claim, including authentication guards, query filters, and data-sanitization logic. Every piece of evidence must be a cited file and line. The skill requires that auditors "read the code that enforces (or fails to enforce) each claim" and treat "Evidence [as] a cited file and line."

Step 3: Compare Claim to Code

For each documented rule, the auditor asks: Does an enforcement point actually implement it on every relevant path? Comments claiming something is "handled upstream" are ignored unless a concrete check is located. The methodology explicitly instructs practitioners to "distrust comments" and verify that enforcement points actually implement the claimed protections.

Step 4: Classify Mismatches

Only mismatches that cross a trust, cost, data, or tenant boundary are considered meaningful. Cosmetic issues, such as typos in comments, are discarded. According to the skill definition, "a mismatch matters when crossing it lets a real actor reach data, money, infrastructure, or another tenant."

Step 5: Produce Concrete Findings

Each finding must record four specific elements: the quoted intent, the cited code evidence (or lack thereof), the attacker/victim scenario, and a specific remediation. If both sides cannot be cited, the issue is logged as an investigation point rather than a defect. The skill mandates that "every finding names: the documented intent, the implemented reality, the attacker and victim, and the concrete fix."

Why Intent-to-Code Verification Matters

This verification method addresses three critical quality dimensions:

• Security – Detects undocumented enforcement gaps, such as permissions declared in documentation but never checked in code, which can lead to privilege escalation.
• Correctness – Identifies code that enforces rules no longer present in documentation, indicating stale or divergent documentation.
• Compliance – Provides auditors with traceable evidence mapping policy statements directly to implementation lines, satisfying regulatory requirements.

Running the Audit in Practice

The skill is designed to integrate into CI/CD pipelines through the pm-ai-shipping command suite. Below is a minimal example of invoking the audit from Python:

import subprocess

# Run the intended-vs-implemented audit from repository root

result = subprocess.run(
    ["pm", "ai-shipping", "audit", "--skill", "intended-vs-implemented"],
    capture_output=True,
    text=True,
)

print(result.stdout)  # Human-readable report of intent-vs-implementation gaps

The command executes the five-step method automatically, producing structured output such as:

[HIGH] permissions.md: "Only admin may delete user"
    → src/users/delete.py: line 23 – missing admin check
    → Attacker: regular user, Victim: any user data
    → Fix: add `if not request.user.is_admin: raise PermissionError`

Key Repository Files

The following files in phuryn/pm-skills define the skill and its integration:

• pm-ai-shipping/skills/intended-vs-implemented/SKILL.md – Full specification of the audit method, classification rules, and output format.
• pm-ai-shipping/commands/ship-check.md – Example command implementation for running the skill within shipping pipelines.
• pm-ai-shipping/skills/shipping-artifacts/SKILL.md – Describes how to produce documentation artifacts (e.g., permissions.md) consumed by the audit.
• README.md – Overview of the PM-Skills ecosystem and workflow integration.

Summary

• The intended-vs-implemented skill audits code against documentation by comparing documented claims to concrete enforcement points in the source.
• It follows a five-step method: establish intent, gather evidence, compare claim to code, classify mismatches by security impact, and produce actionable findings.
• Only mismatches crossing trust, cost, data, or tenant boundaries are reported, filtering out cosmetic drift.
• Findings must include cited file-and-line references, attacker/victim scenarios, and specific remediation steps.
• The skill is implemented in the phuryn/pm-skills repository and integrated via the pm-ai-shipping command suite.

Frequently Asked Questions

What makes the intended-vs-implemented skill different from static analysis?

Static analysis tools detect syntactic patterns and generic bug classes without understanding business intent. The intended-vs-implemented skill requires human or AI auditors to map specific documentation claims (e.g., "only admins may delete users") to exact enforcement locations in code, surfacing semantic gaps that linters cannot identify.

How should documentation be structured for this audit?

Documentation should reside in markdown files within a /documentation/ folder, with clear statements of rules, permissions, and architectural constraints. Each claim should be explicit enough to verify against code, such as "User data must be filtered by tenant_id in all database queries" rather than vague guidelines.

What qualifies as a "meaningful" mismatch?

A mismatch is meaningful only if it crosses a boundary that exposes data, financial resources, infrastructure, or tenant isolation to unauthorized actors. Cosmetic issues like outdated comments or formatting inconsistencies are explicitly excluded from findings unless they indicate a deeper enforcement gap.

Can this skill audit AI-generated code specifically?

Yes, the skill is designed to validate AI-generated code by treating the AI's output as the implementation and comparing it against human-written documentation. This reveals cases where the AI has hallucinated security checks or omitted required enforcement logic that the documentation mandates.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}