Playwright Test Generation and Fixing Flaky Tests with Claude Skills

The Playwright Pro skill in the alirezarezvani/claude-skills repository turns natural language prompts into production-ready Playwright tests and automatically diagnoses flaky failures using a structured taxonomy, eliminating manual boilerplate writing and repetitive debugging cycles.

The alirezarezvani/claude-skills repository hosts a modular library of AI agent skills, with the Playwright Pro skill located under engineering-team/playwright-pro providing a complete toolkit for Playwright test generation and fixing flaky tests. This production-grade module integrates directly with Claude Code through declarative markdown workflows, enabling teams to bootstrap entire test suites and remediate intermittent CI failures without touching configuration files manually.

Understanding the Playwright Pro Architecture

The Playwright Pro skill follows a plug-in model where each capability is defined in markdown files discovered by the agent at runtime. The architecture centers on two primary workflows—generate and fix—orchestrated through slash commands defined in CLAUDE.md.

Key components include:

• CLAUDE.md – The agent manifest that maps slash commands (/pw:generate, /pw:fix, etc.) to specific skill workflows located in the skills/ directory.
• skills/generate/SKILL.md – Defines the test generation pipeline, from parsing intent to emitting verified .spec.ts files.
• skills/fix/SKILL.md – Implements the flaky-test remediation workflow using structured diagnosis.
• skills/fix/flaky-taxonomy.md – A diagnostic reference that maps failure symptoms to root causes and remediation strategies.
• templates/ – A library of 55 parametrizable markdown templates for common testing scenarios (auth, CRUD, checkout).

When invoked, the skill uses an Explore sub-agent to analyze the host project's playwright.config.ts, existing fixtures, and page objects, ensuring generated code aligns with existing conventions rather than hard-coded assumptions.

Generating Tests with /pw:generate

Test generation begins when a developer issues the /pw:generate command followed by a natural language specification. According to the workflow defined in engineering-team/playwright-pro/skills/generate/SKILL.md, the agent executes a four-phase process:

1. Intent Parsing – The agent extracts testing requirements from the user's description (e.g., "user can log in with email and password").
2. Codebase Exploration – The Explore sub-agent reads playwright.config.ts and existing page objects to identify URL patterns, selector strategies, and fixture availability.
3. Template Selection – Based on the domain (authentication, checkout, etc.), the agent selects a markdown template from the templates/ folder (e.g., templates/auth/login.md) and replaces placeholders like {{selectors}} and {{url}} with concrete values derived from the source code.
4. Verification – The emitted .spec.ts file is immediately executed to confirm it passes before the agent presents the final output.

The generated tests follow Playwright best practices strictly: they use web-first assertions (expect(...).toBeVisible()), prefer locator strategies by priority (role → label → text), and include proper Arrange-Act-Assert structuring.

Example: Generating an Authentication Test

# In a Claude Code session

/pw:generate "user can log in with email and password"

The resulting login.spec.ts, rendered from templates/auth/login.md, contains:

import { test, expect } from '@playwright/test';

test.describe('Authentication – Login', () => {
  test('should allow a user to log in with email and password', async ({
    page,
  }) => {
    // Arrange
    await page.goto('/login');
    await expect(page.getByRole('heading', { name: 'Sign in' })).toBeVisible();

    // Act
    await page.getByLabel('Email').fill('user@example.com');
    await page.getByLabel('Password').fill('Secret123!');
    await page.getByRole('button', { name: 'Log in' }).click();

    // Assert
    await expect(page).toHaveURL('/dashboard');
    await expect(page.getByRole('heading', { name: 'Welcome' })).toBeVisible();
  });
});

Fixing Flaky Tests with /pw:fix

Flaky test remediation is handled by the workflow defined in engineering-team/playwright-pro/skills/fix/SKILL.md. When a developer runs /pw:fix pointing to a failing test file, the agent implements a systematic debugging protocol:

1. Failure Reproduction – The agent runs the suspect test with --repeat-each=10 to confirm flakiness and capture the failure rate.
2. Symptom Classification – The agent loads engineering-team/playwright-pro/skills/fix/flaky-taxonomy.md, which categorizes failures into four specific buckets:
   • Timing/Async – Network races, premature assertions before DOM updates
   • Test Isolation – Shared state leakage between tests, missing fixture cleanup
   • Environment – Browser-specific behaviors, viewport dependencies
   • Infrastructure – CI resource constraints, unstable Grid connections
3. Targeted Remediation – Based on the classification, the agent applies domain-specific fixes (e.g., replacing waitForTimeout with waitForResponse, injecting per-test fixtures).
4. Prevention Layer – After verification, the agent appends CI recommendations such as enabling trace: 'on-first-retry' or configuring retries: 2 in playwright.config.ts.

The verification loop requires 10/10 passes (--repeat-each=10) before the fix is considered complete, ensuring the solution is robust rather than coincidental.

Example: Repairing a Race Condition

/pw:fix e2e/checkout.spec.ts "should complete purchase"

Diagnostic flow executed by the agent:

1. Initial run: 6/10 passes indicates intermittent failure.
2. Taxonomy lookup: Symptoms match Timing/Async (network race condition).
3. Fix applied: Replace arbitrary wait with explicit response waiter:

// Before (flaky)
await page.waitForTimeout(2000);
await page.click('button:has-text("Pay now")');

// After (stable)
await Promise.all([
  page.waitForResponse('**/api/checkout**'),
  page.click('button:has-text("Pay now")'),
]);
await expect(page).toHaveURL(/order-confirmation/);

4. Verification: Re-run with --repeat-each=10 achieves 10/10 passes.
5. Output: The agent provides a diff showing the line changes and recommends adding retries: 2 to the CI configuration.

Extending with Custom Templates

The template engine scans the templates/ directory at runtime, allowing teams to add domain-specific variants without modifying core skill logic. Creating a new file at templates/custom/report.md with appropriate placeholders makes it immediately available to the /pw:generate command.

/pw:generate "admin can export a usage report"

The skill automatically discovers the new template and applies the same placeholder substitution and verification workflow used for built-in templates.

Key Files in the Repository

File	Purpose
engineering-team/playwright-pro/README.md	Human-readable installation guide and command reference
engineering-team/playwright-pro/CLAUDE.md	Agent-side manifest mapping slash commands to skill names
engineering-team/playwright-pro/skills/generate/SKILL.md	Declarative workflow for test generation
engineering-team/playwright-pro/skills/fix/SKILL.md	Declarative workflow for flaky test diagnosis and repair
engineering-team/playwright-pro/skills/fix/flaky-taxonomy.md	Diagnostic taxonomy mapping symptoms to fixes
engineering-team/playwright-pro/templates/	55 parametrizable markdown templates for test scaffolding

Summary

• Playwright test generation and fixing flaky tests are automated through the Playwright Pro skill using nine slash commands that integrate with Claude Code.
• The generation workflow in skills/generate/SKILL.md produces type-safe .spec.ts files from 55 customizable templates, respecting existing project conventions via the Explore sub-agent.
• Flaky test remediation follows a deterministic taxonomy defined in flaky-taxonomy.md, categorizing failures into Timing/Async, Test Isolation, Environment, or Infrastructure buckets before applying targeted fixes.
• All fixes require verification via --repeat-each=10 to ensure stability, with automatic CI recommendations for trace capture and retry configuration.
• The plug-in architecture allows extension through simple markdown files, requiring no additional dependencies in the host project.

Frequently Asked Questions

How does the Playwright Pro skill access my project configuration?

The skill uses a built-in Explore sub-agent to read playwright.config.ts, existing test fixtures, and page object files before generating or fixing tests. This ensures all emitted code follows your project's existing selector strategies, base URLs, and fixture patterns without hard-coded assumptions.

Can I use custom templates for my specific domain?

Yes. The template engine scans the engineering-team/playwright-pro/templates/ directory at runtime. Adding a new markdown file with placeholders (e.g., {{selectors}}, {{url}}) makes it immediately available to the /pw:generate command. The skill automatically substitutes placeholders with values derived from your component source or user story.

What constitutes a "fixed" flaky test in the remediation workflow?

A test is considered fixed only when it achieves 10 consecutive passes using Playwright's --repeat-each=10 flag. This verification loop, defined in skills/fix/SKILL.md, prevents accepting coincidental fixes. Additionally, the skill appends prevention recommendations like enabling trace: 'on-first-retry' or configuring retries: 2 in your CI pipeline.

Is the skill safe to run in CI environments?

Yes. The skill operates declaratively and only executes the standard Playwright CLI (npx playwright) already present in your project. It does not run arbitrary shell commands or install dependencies, making it safe for automated pipelines while maintaining reproducibility across local and CI environments.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}