What Tests Are Included in pm-skills? QA Scenarios, A/B Analysis, and Coverage Mapping

The pm-skills repository includes comprehensive testing capabilities spanning QA test-scenario generation, statistical A/B test evaluation, and AI-driven test-coverage mapping.

The pm-skills marketplace equips product managers with specialized tools for creating, analyzing, and auditing tests throughout the development lifecycle. These capabilities cover everything from drafting detailed QA scenarios to evaluating statistical significance in experiments and identifying gaps in test coverage. Understanding what tests are included in pm-skills enables teams to implement quality assurance at every stage of product delivery.

QA Test Scenario Generation

The repository provides dedicated skills for transforming user stories into executable test plans.

Test Scenario Generation Skill

The test-scenarios skill, defined in [pm-execution/skills/test-scenarios/SKILL.md](https://github.com/phuryn/pm-skills/blob/main/pm-execution/skills/test-scenarios/SKILL.md), generates detailed QA test scenarios from user stories. It constructs comprehensive test cases including objectives, starting conditions, user roles, step-by-step actions, and expected outcomes.

Slash Command Usage

Product managers invoke this capability through the /test-scenarios command, documented in [pm-execution/commands/test-scenarios.md](https://github.com/phuryn/pm-skills/blob/main/pm-execution/commands/test-scenarios.md).

/test-scenarios
PRODUCT: MyWebApp
USER_STORY: "As a shopper, I want to add items to a wishlist so I can purchase them later."
CONTEXT: Wishlist must persist across sessions and respect privacy settings.

Result:

Test Scenario: Add item to wishlist
Test Objective: Verify that an item can be added and appears in the wishlist view.
Starting Conditions:
  - User is logged in.
  - Wishlist is empty.
User Role: Shopper
Test Steps:
  1. Navigate to a product page → “Add to wishlist” button is visible.
  2. Click “Add to wishlist” → Item appears in the “Saved items” list.
  …
Expected Outcomes:
  - Wishlist shows the newly added item.
  - Item persists after logout/login.

Data Analytics Testing

The analytics module provides tools for evaluating experiments and user behavior through statistical methods.

A/B Test Analysis

The ab-test-analysis skill in [pm-data-analytics/skills/ab-test-analysis/SKILL.md](https://github.com/phuryn/pm-skills/blob/main/pm-data-analytics/skills/ab-test-analysis/SKILL.md) interprets experiment results, checks statistical significance, calculates required sample sizes, and recommends ship, extend, or stop actions. Access this via the /analyze-test command in [pm-data-analytics/commands/analyze-test.md](https://github.com/phuryn/pm-skills/blob/main/pm-data-analytics/commands/analyze-test.md).

/analyze-test
EXPERIMENT: Checkout‑flow‑v2
DATA: attach checkout-ab-results.csv

Result:

✔ Sample size sufficient (95 % confidence, 2 % MDE)
✔ Statistically significant lift: 4.3 % increase in conversion
🚀 Recommendation: Ship to production

Cohort Analysis

The cohort-analysis skill, located in [pm-data-analytics/skills/cohort-analysis/SKILL.md](https://github.com/phuryn/pm-skills/blob/main/pm-data-analytics/skills/cohort-analysis/SKILL.md), produces retention curves, feature-adoption trends, and engagement metrics broken down by user cohorts. Invoke this analysis using the /analyze-cohorts command.

AI Shipping Test Coverage and Audits

The AI shipping module focuses on validating code quality and coverage before production deployment.

Test Coverage Mapping

The /derive-tests command, defined in [pm-ai-shipping/commands/derive-tests.md](https://github.com/phuryn/pm-skills/blob/main/pm-ai-shipping/commands/derive-tests.md), extracts existing test artifacts, highlights missing coverage areas, and proposes new tests before merge.

/derive-tests
REPO: https://github.com/example/my‑service

Result:

✅ Existing tests: 12 unit, 4 integration
❓ Missing coverage:
  - Authorization checks for admin endpoints
  - Edge‑case handling of malformed JSON payloads
🛠 Suggested new tests: 3 unit, 2 integration

Security and Performance Audits

Static auditing capabilities include:

• /security-audit-static: Scans code for security gaps and vulnerabilities (see [pm-ai-shipping/commands/security-audit-static.md](https://github.com/phuryn/pm-skills/blob/main/pm-ai-shipping/commands/security-audit-static.md))
• /performance-audit-static: Identifies performance bottlenecks by cross-referencing documented intent (see [pm-ai-shipping/commands/performance-audit-static.md](https://github.com/phuryn/pm-skills/blob/main/pm-ai-shipping/commands/performance-audit-static.md))

Summary

• QA Testing: Generate detailed test scenarios from user stories using /test-scenarios in [pm-execution/skills/test-scenarios/SKILL.md](https://github.com/phuryn/pm-skills/blob/main/pm-execution/skills/test-scenarios/SKILL.md)
• Analytics Testing: Evaluate A/B tests with /analyze-test and cohort retention with /analyze-cohorts via pm-data-analytics/skills/
• Coverage Mapping: Identify test gaps using /derive-tests in [pm-ai-shipping/commands/derive-tests.md](https://github.com/phuryn/pm-skills/blob/main/pm-ai-shipping/commands/derive-tests.md)
• Static Audits: Run security and performance checks before shipping using commands in pm-ai-shipping/commands/

Frequently Asked Questions

How do I generate QA test scenarios in pm-skills?

Use the /test-scenarios slash command with your product name, user story, and context parameters. This invokes the skill defined in [pm-execution/skills/test-scenarios/SKILL.md](https://github.com/phuryn/pm-skills/blob/main/pm-execution/skills/test-scenarios/SKILL.md) to produce structured test cases with steps, conditions, and expected outcomes.

What commands are available for A/B test analysis?

The /analyze-test command runs the ab-test-analysis skill to check statistical significance and recommend actions. For cohort-based retention analysis, use /analyze-cohorts. Both are located in the pm-data-analytics/commands/ directory.

How does pm-skills identify missing test coverage?

The /derive-tests command maps existing test artifacts against your repository code to highlight uncovered areas. According to [pm-ai-shipping/commands/derive-tests.md](https://github.com/phuryn/pm-skills/blob/main/pm-ai-shipping/commands/derive-tests.md), it categorizes current tests by type (unit, integration) and suggests specific new tests for uncovered endpoints and edge cases.

Can pm-skills perform automated security audits?

Yes. The /security-audit-static command in [pm-ai-shipping/commands/security-audit-static.md](https://github.com/phuryn/pm-skills/blob/main/pm-ai-shipping/commands/security-audit-static.md) performs static analysis to detect security gaps. For performance bottlenecks, use /performance-audit-static from the same directory.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}