# How to Run Tests in the different-ai/openwork Repository

> Learn how to run tests in the different-ai/openwork repository using pnpm. Execute end-to-end tests with pnpm test:e2e and spec-driven evaluations with pnpm evals:test.

- Repository: [Different AI/openwork](https://github.com/different-ai/openwork)
- Tags: how-to-guide
- Published: 2026-08-04

---

**Use pnpm to execute test scripts defined in the root [`package.json`](https://github.com/different-ai/openwork/blob/main/package.json), including `pnpm test:e2e` for end-to-end tests and `pnpm evals:test` for spec-driven evaluations.**

The openwork project is a monorepo managed with pnpm that contains multiple test suites across different packages. Testing is organized through npm scripts in [`package.json`](https://github.com/different-ai/openwork/blob/main/package.json) and relies on the custom `@openwork/testkit` framework for end-to-end validation. This guide shows you exactly how to execute each test type based on the actual source code structure.

## Prerequisites: Install Dependencies

Before running any tests, install all workspace dependencies at the repository root.

```bash
pnpm install

```

The repository strictly requires pnpm. Using npm or yarn is explicitly unsupported according to the codebase configuration.

## Available Test Scripts in package.json

All test commands are centralized in the root [`package.json`](https://github.com/different-ai/openwork/blob/main/package.json) scripts section. Here are the primary test scripts you need to know.

### Run Full End-to-End Test Suite

The `test:e2e` script executes the complete e2e test suite for the desktop application (`@openwork/app`).

```bash
pnpm test:e2e

```

This script internally runs `pnpm --filter @openwork/app test:e2e`, which invokes Vitest with the appropriate configuration for the app package.

### Run OpenWork Evals (Spec-Driven Tests)

The `evals:test` script executes all evaluation specifications across the `evals/` workspace.

```bash
pnpm evals:test

```

These tests live in `evals/specs/` and use the [`.slow.test.ts`](https://github.com/different-ai/openwork/blob/main/.slow.test.ts) suffix. They produce a test-kit evidence tape that proves features work on real OpenWork instances. The script entry point is `evals/runner/run.mjs`, which sets up the test-kit environment before execution.

### Run Large-Scale Admin Tests

For testing the Den API server under load, use the admin-scale script.

```bash
pnpm test:admin-scale

```

### Run Package-Specific Unit Tests

Individual packages expose their own test scripts for targeted validation:

```bash
pnpm test:health      # App health checks

pnpm test:sessions    # Session handling tests

pnpm test:events      # Event processing tests

```

## Test File Locations and Patterns

Understanding where tests live helps you run specific files directly.

| Pattern | Location | Purpose |
| --- | --- | --- |
| `*.slow.test.ts` | `evals/specs/` | End-to-end specs with real browser automation |
| `*.test.ts` | `packages/*/tests/` | Unit and integration tests per package |

A typical e2e spec in [`evals/specs/example.slow.test.ts`](https://github.com/different-ai/openwork/blob/main/evals/specs/example.slow.test.ts) looks like this:

```typescript
import { test } from '@openwork/testkit';

test('example flow', async ({ page }) => {
  await page.goto('https://example.com');
  // …interact with the UI, assert expectations…
});

```

## Running Individual Test Files

To focus on a single spec without executing the full suite, invoke Vitest directly through pnpm:

```bash
pnpm --filter @openwork/app exec vitest evals/specs/example.slow.test.ts

```

This bypasses the script wrappers and runs only the specified file with Vitest's watch mode and filtering capabilities.

## Environment Considerations

- **Default configuration**: Most test scripts work out-of-the-box with values defined in [`package.json`](https://github.com/different-ai/openwork/blob/main/package.json)
- **Development mode**: Some tests may require `OPENWORK_DEV_MODE=1` for specific dev behaviors
- **Worktree isolation**: Start an isolated worktree with `pnpm dev:worktree` before testing if you need profile-targeted test runs

## Key Source Files

These files contain the actual test configuration and implementation:

- [`package.json`](https://github.com/different-ai/openwork/blob/main/package.json) — Central script definitions for all test commands
- `evals/runner/run.mjs` — CLI entry point for `evals:*` commands
- `evals/specs/*.slow.test.ts` — Representative e2e spec files
- `packages/app/tests/` — Desktop application unit and integration tests

## Summary

- **pnpm is mandatory** — install with `pnpm install` before any testing
- **`pnpm test:e2e`** — runs the full desktop app end-to-end suite
- **`pnpm evals:test`** — executes all spec-driven evaluations in `evals/specs/`
- **Test files use [`.slow.test.ts`](https://github.com/different-ai/openwork/blob/main/.slow.test.ts)** for e2e and [`.test.ts`](https://github.com/different-ai/openwork/blob/main/.test.ts) for unit tests
- **Run individual files** with `pnpm --filter @openwork/app exec vitest <path>`

## Frequently Asked Questions

### What test framework does openwork use?

The openwork repository uses **Vitest** as the underlying test runner, wrapped with a custom `@openwork/testkit` package for end-to-end browser automation. According to the source code in [`package.json`](https://github.com/different-ai/openwork/blob/main/package.json) and `evals/runner/run.mjs`, pnpm scripts coordinate test execution across the monorepo while the testkit provides page fixture and evidence recording capabilities.

### Can I use npm or yarn instead of pnpm?

No. The codebase explicitly requires pnpm and defines workspace commands that depend on pnpm-specific features like `--filter`. Attempting to use npm or yarn will fail because the repository structure relies on pnpm workspaces and hoisting behavior configured in [`pnpm-workspace.yaml`](https://github.com/different-ai/openwork/blob/main/pnpm-workspace.yaml).

### Where are the actual test files located?

End-to-end specs are in `evals/specs/` with the [`.slow.test.ts`](https://github.com/different-ai/openwork/blob/main/.slow.test.ts) suffix, while package-specific tests live in `packages/[package-name]/tests/`. The `evals/specs/` directory contains the comprehensive evaluation suite that validates OpenWork features against real-running instances.

### How do I run tests in a specific package only?

Use pnpm's filter flag to target a single workspace package. For example: `pnpm --filter @openwork/app test` or `pnpm --filter @openwork/den-api test:admin-scale`. The filter syntax ensures only that package's dependencies get installed and only its test configuration loads.