How to Use Shipping-Artifacts to Find Gaps Between Documented Intent and Code Implementation

The shipping-artifacts skill in phuryn/pm-skills creates a durable documentation baseline that captures application intent, which the intended-vs-implemented skill then systematically compares against actual source code to flag security-critical mismatches at trust, data, cost, or tenant boundaries.

The shipping-artifacts methodology provides a concrete framework for verifying that AI-generated applications match their design specifications. By treating documentation as a first-class contract rather than an afterthought, this approach enables automated verification of implementation fidelity. In the phuryn/pm-skills repository, the combination of artifact generation and gap analysis creates a closed-loop system for maintaining code accuracy.

What Are Shipping-Artifacts?

The shipping-artifacts skill defines a minimal, durable set of documentation that captures the intent of an AI-generated application. According to pm-ai-shipping/skills/shipping-artifacts/SKILL.md, these artifacts serve as the "intended-state" baseline stored under /documentation/.

The core artifact set includes:

• architecture.md – System structure and component relationships
• flows.md – Business logic and user journey sequences
• permissions.md – Authorization rules and access controls
• variables.md – Configuration and environment variables
• tests.md – Verification criteria (auto-generated)

Conditional artifacts are generated only when applicable:

• emails.md – Notification templates and triggers
• cron.md – Scheduled job specifications
• seo.md – Search optimization requirements
• automation.md – Background processing rules

The Gap Analysis Workflow

Step 1 – Generate the Intent Artifacts

Run the /document-app command (or pm-skills document-app /path/to/your/app) to produce the core documentation set. This creates the /documentation/ directory containing the files defined in the shipping-artifacts specification.

These files establish the source of truth for what the system should do, providing the concrete baseline against which implementation will be measured.

Step 2 – Derive the Verification Map

Invoke /derive-tests to synthesize tests.md, which ties every documented rule to existing test coverage. As implemented in pm-ai-shipping/commands/derive-tests.md, this command maps each intent claim to:

1. An existing test
2. A proposed test
3. A gap (no test coverage)

Step 3 – Run the Gap Analysis

Execute the intended-vs-implemented skill via /ship-check or directly with opencode run intended-vs-implemented. According to pm-ai-shipping/skills/intended-vs-implemented/SKILL.md, this process performs four critical operations:

1. Reads each intent claim from the documentation set
2. Locates the exact code construct enforcing that claim, citing file and line numbers
3. Compares the documented intent against the implementation
4. Classifies mismatches as relevant only when exposing boundary-crossing risks (unauthorized data access, unguarded background jobs, etc.)

The analysis emits structured findings including the quoted intent, code evidence, attacker/victim scenarios, and concrete remediation steps.

Step 4 – Iterate Until Clean

For every gap reported, update either the documentation or the code, regenerate the artifacts, and re-run the analysis. This disciplined loop ensures that shipping-artifacts remain an actively maintained contract between design intent and implementation reality.

Practical Implementation

Run the following commands to audit your application:

# Generate the intent documentation (core + conditional docs)

pm-skills document-app /path/to/your/app

# Create the test-coverage map from docs and existing tests

pm-skills derive-tests /path/to/your/app

# Execute the gap analysis comparing docs against code

pm-skills ship-check --mode intended-vs-implemented \
    --docs ./documentation \
    --src ./src

The output identifies specific divergences:

[Gap] permissions.md → src/auth/check_permission.py:45
   Intent: "admin role may delete any user"
   Implementation: No explicit admin check; generic role check only.
   Impact: Unauthorized delete possible for non-admin users.
   Recommendation: Add admin-only guard in delete endpoint.

Key Source Files in phuryn/pm-skills

Understanding the repository structure helps customize the analysis:

• pm-ai-shipping/skills/shipping-artifacts/SKILL.md – Defines the core and conditional documentation set (the intent baseline)
• pm-ai-shipping/skills/intended-vs-implemented/SKILL.md – Describes the systematic gap-analysis method that consumes the artifacts
• pm-ai-shipping/commands/document-app.md – CLI command that auto-generates documentation artifacts from code
• pm-ai-shipping/commands/derive-tests.md – CLI command producing tests.md linking docs to test coverage
• pm-ai-shipping/commands/ship-check.md – Orchestrates the full audit, invoking both artifact generation and gap analysis

Summary

• Shipping-artifacts create a durable, minimal documentation set that captures application intent in architecture.md, flows.md, permissions.md, and other core files.
• The /document-app command generates these artifacts under /documentation/, establishing a verifiable source of truth.
• /derive-tests synthesizes tests.md to map every documented claim to test coverage, identifying untested intent.
• The intended-vs-implemented skill compares documentation against actual source code, flagging mismatches only when they cross security-sensitive boundaries.
• This workflow transforms static documentation into an active contract, enabling reliable security, performance, and compliance audits.

Frequently Asked Questions

What specific files does the document-app command generate?

The command produces a core set including architecture.md, flows.md, permissions.md, variables.md, and tests.md. It also conditionally generates emails.md, cron.md, seo.md, and automation.md when those features are detected in your codebase. All files reside under /documentation/ and follow the schema defined in pm-ai-shipping/skills/shipping-artifacts/SKILL.md.

How does the tool determine which gaps are relevant versus false positives?

The intended-vs-implemented skill classifies mismatches as relevant only when they expose a boundary-crossing risk. This includes unauthorized data access, unguarded background jobs, cost-impacting operations, or tenant isolation violations. Documentation that drifts from implementation without crossing these trust, data, cost, or tenant boundaries is noted but not flagged as a critical gap.

Can I run the gap analysis without generating new artifacts?

Yes. If you have existing documentation in /documentation/, you can run pm-skills ship-check --mode intended-vs-implemented directly against your source directory. However, regenerating artifacts ensures the intent baseline reflects the current state of your design documents before comparison.

What boundaries does the gap analysis check for?

The analysis specifically watches for four boundary types: trust (authentication and authorization controls), data (storage and access patterns), cost (resource-intensive operations), and tenant (multi-tenant isolation). Any divergence between documented intent and code implementation that crosses one of these boundaries triggers a structured finding with remediation steps.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}