# How to Design Effective Experiments for Testing Product Assumptions

> Learn to design effective experiments for testing product assumptions with a systematic five-step framework. Rapidly validate ideas using real behavior metrics, not opinions. Explore the phuryn/pm-skills repository.

- Repository: [Pawel Huryn/pm-skills](https://github.com/phuryn/pm-skills)
- Tags: tutorial
- Published: 2026-06-13

---

**TLDR:** The `phuryn/pm-skills` repository provides a systematic, markdown-based skill that guides product managers through a five-step framework to design low-effort experiments, enabling rapid validation of assumptions using real behavioral metrics rather than opinions.

Designing effective experiments for testing product assumptions is essential for de-risking product decisions without wasting engineering resources. The PM Skills Marketplace open-source repository ships a dedicated `brainstorm-experiments-existing` skill that implements a structured framework for experiment design. This guide walks through the exact methodology defined in the source code, including practical prompts you can deploy immediately with any AI assistant.

## The 5-Step Experiment Design Framework

The core logic resides in [`pm-product-discovery/skills/brainstorm-experiments-existing/SKILL.md`](https://github.com/phuryn/pm-skills/blob/main/pm-product-discovery/skills/brainstorm-experiments-existing/SKILL.md), which structures the process into five distinct phases.

### Step 1: Clarify the Idea and Assumptions

Before selecting experiment types, capture the feature idea, target user, and specific hypotheses requiring validation. The skill consumes this context in its *Context* section to tailor subsequent recommendations to your specific scenario.

### Step 2: Choose Experiment Types

Select from the predefined experiment toolbox:

- **First-click prototypes** – Test initial user engagement with a mocked interface.
- **Fake-door tests** – Measure intent by presenting a non-functional feature option.
- **Technical spikes** – Validate technical feasibility with time-boxed exploration.
- **A/B tests** – Compare variants with controlled exposure.
- **Wizard-of-Oz experiments** – Simulate full functionality while humans handle the backend manually.
- **Survey-based validation** – Gather quantitative data on user preferences.

### Step 3: Apply Key Principles

The skill enforces three non-negotiable principles:

- Measure real behavior, not opinion.
- Mitigate risk (e.g., limit exposure in A/B tests).
- Maximize learning per unit of effort.

### Step 4: Flesh Out Each Experiment

For every assumption, define four specific fields:

- **Assumption** – What you believe is true.
- **Experiment** – Concrete execution steps (e.g., "launch a fake-door button for Feature X").
- **Metric** – Measurable user action (e.g., click-through rate).
- **Success Threshold** – The numeric value that confirms the assumption (e.g., "≥ 12% CTR").

### Step 5: Present Results

Render a markdown table or structured document so the team can review and iterate. The skill recommends a clear table layout for easy scanning.

## Practical Implementation: Using the `brainstorm-experiments-existing` Skill

Because the skill is pure markdown, it works with any AI assistant that reads [`SKILL.md`](https://github.com/phuryn/pm-skills/blob/main/SKILL.md) files (Claude, Gemini, OpenCode, etc.). No code execution is required—only the structured guidance it contains.

### Example 1: Simple One-Assumption Experiment

Trigger the skill by describing your assumption and requesting the specific output format:

```markdown
Design experiments for the following assumption:

**Assumption:** Users will click "Try Free" on the landing page if we show a "30-day trial" badge.

Please output a markdown table with columns: Assumption, Experiment, Metric, Success Threshold.

```

**Expected Output:**

| Assumption | Experiment | Metric | Success Threshold |
|------------|------------|--------|-------------------|
| Users will click "Try Free" when a "30-day trial" badge is displayed. | Add a temporary banner with the badge on the homepage (A/B test). | Click-through rate on "Try Free". | ≥ 12% CTR (baseline 8%). |

### Example 2: Full Discovery Chain with `/discover`

For a complete discovery workflow, use the `/discover` command defined in [`pm-product-discovery/commands/discover.md`](https://github.com/phuryn/pm-skills/blob/main/pm-product-discovery/commands/discover.md). This orchestrates a four-step chain:

```markdown
/discover
AI-powered meeting summarizer for remote teams

```

The command executes sequentially:

1. `brainstorm-ideas-existing` – Generate ideas for the summarizer.
2. `identify-assumptions-existing` – Surface risky assumptions (e.g., "users will tolerate 2-minute latency").
3. `prioritize-assumptions` – Rank assumptions on the Impact × Risk matrix.
4. `brainstorm-experiments-existing` – Produce the experiment table for each high-impact/high-risk assumption.

## Repository Architecture and Key Files

Understanding the file structure helps you customize the framework or integrate it into custom workflows.

- **[`pm-product-discovery/skills/brainstorm-experiments-existing/SKILL.md`](https://github.com/phuryn/pm-skills/blob/main/pm-product-discovery/skills/brainstorm-experiments-existing/SKILL.md)** – Contains the core experiment-design logic, step-by-step guide, and output format specifications.
- **[`pm-product-discovery/skills/prioritize-assumptions/SKILL.md`](https://github.com/phuryn/pm-skills/blob/main/pm-product-discovery/skills/prioritize-assumptions/SKILL.md)** – Provides the Impact × Risk matrix used to determine which assumptions need experiments first.
- **[`pm-product-discovery/commands/discover.md`](https://github.com/phuryn/pm-skills/blob/main/pm-product-discovery/commands/discover.md)** – Defines the command that chains the full discovery workflow from ideation to experiment design.
- **[`CLAUDE.md`](https://github.com/phuryn/pm-skills/blob/main/CLAUDE.md)** – Definitive agent guidance specifying how AI systems load and invoke skills.

## Summary

- The `brainstorm-experiments-existing` skill in `phuryn/pm-skills` provides a five-step framework for designing product experiments.
- Each experiment requires four defined fields: Assumption, Experiment, Metric, and Success Threshold.
- The toolbox includes six experiment types: first-click prototypes, fake-door tests, technical spikes, A/B tests, Wizard-of-Oz, and survey-based validation.
- Use the `/discover` command to run the full discovery chain: brainstorming ideas, identifying assumptions, prioritizing them, and designing experiments.
- All skills are pure markdown, making them compatible with Claude, Gemini, OpenCode, and other AI assistants without requiring code execution.

## Frequently Asked Questions

### What is the difference between a fake-door test and a Wizard-of-Oz experiment?

A **fake-door test** measures intent by presenting a non-functional feature (like a button that shows "coming soon") to gauge click-through rates before building. A **Wizard-of-Oz experiment** appears fully functional to users, but a human manually performs the backend work behind the scenes. Both are supported experiment types in the `brainstorm-experiments-existing` skill according to the repository source code.

### How do I prioritize which assumptions to test first?

Use the `prioritize-assumptions` skill, which plots assumptions on an **Impact × Risk** matrix. High-impact, high-risk assumptions should be validated first through experiments designed by the `brainstorm-experiments-existing` skill. This prioritization logic is defined in [`pm-product-discovery/skills/prioritize-assumptions/SKILL.md`](https://github.com/phuryn/pm-skills/blob/main/pm-product-discovery/skills/prioritize-assumptions/SKILL.md).

### Can I use these experiment design skills without Claude?

Yes. The skills are pure markdown files that work with any AI assistant capable of reading [`SKILL.md`](https://github.com/phuryn/pm-skills/blob/main/SKILL.md) instructions, including Gemini, OpenCode, and custom GPT implementations. The repository's [`CLAUDE.md`](https://github.com/phuryn/pm-skills/blob/main/CLAUDE.md) file provides the standard for agent compatibility but does not restrict usage to Claude alone.

### Where is the success threshold defined in the experiment output?

The **success threshold** is the fourth column in the markdown table generated by the skill. According to [`SKILL.md`](https://github.com/phuryn/pm-skills/blob/main/SKILL.md), this numeric value (e.g., "≥ 12% CTR") determines whether the assumption is validated or invalidated based on the measured metric, allowing teams to make objective go/no-go decisions.