# How to Run Playwright E2E Tests for OpenMAIC: Complete Setup Guide

> Learn to run Playwright E2E tests for OpenMAIC. Install dependencies, configure the server, and execute tests with our complete setup guide.

- Repository: [MAIC/OpenMAIC](https://github.com/THU-MAIC/OpenMAIC)
- Tags: how-to-guide
- Published: 2026-09-10

---

**To run Playwright E2E tests for OpenMAIC, install dependencies with pnpm, ensure the application server runs on port 3002 with the Pro editor enabled, and execute `pnpm exec playwright test` from the repository root.**

The OpenMAIC repository includes a comprehensive Playwright test suite that validates the web UI, agent workbench, and rendering pipeline. This guide explains how to configure and execute Playwright E2E tests for OpenMAIC by referencing the actual implementation in [`playwright.config.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/playwright.config.ts) and the test specifications located in `e2e/tests/`.

## Prerequisites for OpenMAIC E2E Testing

Before executing the test suite, ensure your environment meets the following requirements:

- **Node.js** version 22 or higher
- **pnpm** package manager installed globally

Verify your Node.js version with `node --version` before proceeding.

## Running Playwright E2E Tests Locally

Executing the OpenMAIC Playwright suite locally involves three distinct phases: dependency installation, server configuration, and test execution.

### Install Development Dependencies

Clone the repository and install all required packages using pnpm. The Playwright binaries and browser engines are downloaded automatically during this step.

```bash
git clone https://github.com/THU-MAIC/OpenMAIC.git
cd OpenMAIC
pnpm install

```

### Configure the Application Server

According to [`playwright.config.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/playwright.config.ts), the test runner expects the application to be accessible on **port 3002** with the Pro editor feature explicitly enabled. The configuration automatically handles server startup via the `webServer` block, but you must ensure the environment variables are correctly passed.

The critical environment variables are:
- `PORT=3002`
- `NEXT_PUBLIC_MAIC_EDITOR_ENABLED=true`

Locally, Playwright uses `pnpm dev` to start the server, while CI environments use `pnpm start` for production builds.

### Execute the Test Suite

Run the full Playwright test suite using either the direct command or the package script shortcut defined in [`package.json`](https://github.com/THU-MAIC/OpenMAIC/blob/main/package.json):

```bash

# Direct execution

pnpm exec playwright test

# Or use the shortcut (if defined in package.json)

pnpm test:e2e

```

Playwright will automatically launch Chromium, navigate to `http://localhost:3002`, and execute all specifications in `e2e/tests/` in parallel.

## Understanding the Playwright Configuration

The [`playwright.config.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/playwright.config.ts) file at the repository root defines the test behavior and environment setup. Key configuration properties include:

- **`testDir: './e2e/tests'`** – Specifies the directory containing all [`.spec.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/.spec.ts) test files
- **`baseURL: 'http://localhost:3002'`** – Sets the base URL for all page navigations
- **`webServer.command`** – Dynamically selects between `pnpm dev` (local) and `pnpm start` (CI) based on the environment
- **`env`** – Injects required environment variables including `PORT` and `NEXT_PUBLIC_MAIC_EDITOR_ENABLED`

This configuration ensures consistent test execution across different environments without manual server management.

## Running E2E Tests in CI Pipelines

When executing in continuous integration environments, set the `CI` environment variable to enable production-specific settings:

```bash
CI=1 pnpm exec playwright test

```

The CI-specific adaptations in [`playwright.config.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/playwright.config.ts) include:

- **`forbidOnly: true`** – Prevents tests marked with `.only` from slipping into production
- **`retries: 2`** – Automatically retries failed tests up to two times to handle flakiness
- **`workers: 2`** – Limits parallelism to two workers, optimized for standard 4-core CI runners

These settings ensure reliable test execution in resource-constrained CI environments while maintaining strict quality gates.

## Writing Playwright Tests for OpenMAIC

The OpenMAIC test suite follows the Page Object Model pattern for maintainable test code. Tests are located in `e2e/tests/` while reusable page logic resides in `e2e/pages/`.

### Example Test Specification

The following spec demonstrates navigating from the home page to course generation:

```typescript
// e2e/tests/home-to-generation.spec.ts
import { test, expect, type Page } from '@playwright/test';
import { HomePage } from '../pages/home.page';

test('generate a course from the home page', async ({ page }) => {
  const home = new HomePage(page);
  await home.goto();                         // navigates to baseURL
  await home.fillPrompt('Quantum physics');  // type a topic
  await home.submit();                       // start generation
  await expect(page).toHaveURL(/\/course\/.+/); // redirects to the new course
});

```

### Page Object Implementation

The corresponding page object encapsulates locator strategies and interaction methods:

```typescript
// e2e/pages/home.page.ts
import { type Page, type Locator } from '@playwright/test';

export class HomePage {
  readonly page: Page;
  readonly promptInput: Locator;
  readonly generateBtn: Locator;

  constructor(page: Page) {
    this.page = page;
    this.promptInput = page.locator('#prompt');
    this.generateBtn = page.locator('button:has-text("Generate")');
  }

  async goto() {
    await this.page.goto('/');
  }

  async fillPrompt(text: string) {
    await this.promptInput.fill(text);
  }

  async submit() {
    await this.generateBtn.click();
  }
}

```

This architecture separates test logic from implementation details, making the suite resilient to UI changes.

## Summary

Running Playwright E2E tests for OpenMAIC requires proper environment setup and understanding of the configuration in [`playwright.config.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/playwright.config.ts):

- Install dependencies using `pnpm install` after cloning the repository
- Ensure the application server listens on **port 3002** with `NEXT_PUBLIC_MAIC_EDITOR_ENABLED=true`
- Execute tests via `pnpm exec playwright test` or the `pnpm test:e2e` shortcut
- Use `CI=1` prefix in continuous integration environments to enable production build testing and retry logic
- Organize tests using the Page Object Model pattern with files in `e2e/tests/` and helpers in `e2e/pages/`

## Frequently Asked Questions

### What Node.js version is required to run Playwright tests in OpenMAIC?

OpenMAIC requires **Node.js version 22 or higher** to execute the Playwright test suite. Earlier versions may fail during dependency installation or test execution due to incompatible features in the test runner or application code.

### Why does the Playwright configuration use port 3002 instead of the default Next.js port 3000?

The [`playwright.config.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/playwright.config.ts) explicitly sets `PORT=3002` in the webServer environment block to avoid conflicts with other local development servers and to ensure consistent testing conditions. This configuration prevents interference from other Next.js instances that might be running on the standard port 3000 during development.

### How can I run a specific test file instead of the entire OpenMAIC E2E suite?

To execute a single test file, append the relative path to the Playwright command:

```bash
pnpm exec playwright test e2e/tests/home-to-generation.spec.ts

```

You can also run tests in headed mode for debugging by adding the `--headed` flag, or use `pnpm exec playwright test --ui` to launch the interactive test runner interface.

### Does OpenMAIC support Dockerized Playwright testing?

Yes, the repository includes optional Docker configuration files (`Dockerfile` and [`docker-compose.yml`](https://github.com/THU-MAIC/OpenMAIC/blob/main/docker-compose.yml)) that can containerize the application and test environment. This setup launches the application on port 3002 before executing Playwright tests, ensuring consistent environments across different machines and CI pipelines.