How to Run Tests in the different-ai/openwork Repository

Use pnpm to execute test scripts defined in the root package.json, including pnpm test:e2e for end-to-end tests and pnpm evals:test for spec-driven evaluations.

The openwork project is a monorepo managed with pnpm that contains multiple test suites across different packages. Testing is organized through npm scripts in package.json and relies on the custom @openwork/testkit framework for end-to-end validation. This guide shows you exactly how to execute each test type based on the actual source code structure.

Prerequisites: Install Dependencies

Before running any tests, install all workspace dependencies at the repository root.

pnpm install

The repository strictly requires pnpm. Using npm or yarn is explicitly unsupported according to the codebase configuration.

Available Test Scripts in package.json

All test commands are centralized in the root package.json scripts section. Here are the primary test scripts you need to know.

Run Full End-to-End Test Suite

The test:e2e script executes the complete e2e test suite for the desktop application (@openwork/app).

pnpm test:e2e

This script internally runs pnpm --filter @openwork/app test:e2e, which invokes Vitest with the appropriate configuration for the app package.

Run OpenWork Evals (Spec-Driven Tests)

The evals:test script executes all evaluation specifications across the evals/ workspace.

pnpm evals:test

These tests live in evals/specs/ and use the .slow.test.ts suffix. They produce a test-kit evidence tape that proves features work on real OpenWork instances. The script entry point is evals/runner/run.mjs, which sets up the test-kit environment before execution.

Run Large-Scale Admin Tests

For testing the Den API server under load, use the admin-scale script.

pnpm test:admin-scale

Run Package-Specific Unit Tests

Individual packages expose their own test scripts for targeted validation:

pnpm test:health      # App health checks

pnpm test:sessions    # Session handling tests

pnpm test:events      # Event processing tests

Test File Locations and Patterns

Understanding where tests live helps you run specific files directly.

Pattern	Location	Purpose
*.slow.test.ts	evals/specs/	End-to-end specs with real browser automation
*.test.ts	packages/*/tests/	Unit and integration tests per package

A typical e2e spec in evals/specs/example.slow.test.ts looks like this:

import { test } from '@openwork/testkit';

test('example flow', async ({ page }) => {
  await page.goto('https://example.com');
  // …interact with the UI, assert expectations…
});

Running Individual Test Files

To focus on a single spec without executing the full suite, invoke Vitest directly through pnpm:

pnpm --filter @openwork/app exec vitest evals/specs/example.slow.test.ts

This bypasses the script wrappers and runs only the specified file with Vitest's watch mode and filtering capabilities.

Environment Considerations

• Default configuration: Most test scripts work out-of-the-box with values defined in package.json
• Development mode: Some tests may require OPENWORK_DEV_MODE=1 for specific dev behaviors
• Worktree isolation: Start an isolated worktree with pnpm dev:worktree before testing if you need profile-targeted test runs

Key Source Files

These files contain the actual test configuration and implementation:

• package.json — Central script definitions for all test commands
• evals/runner/run.mjs — CLI entry point for evals:* commands
• evals/specs/*.slow.test.ts — Representative e2e spec files
• packages/app/tests/ — Desktop application unit and integration tests

Summary

• pnpm is mandatory — install with pnpm install before any testing
• pnpm test:e2e — runs the full desktop app end-to-end suite
• pnpm evals:test — executes all spec-driven evaluations in evals/specs/
• Test files use .slow.test.ts for e2e and .test.ts for unit tests
• Run individual files with pnpm --filter @openwork/app exec vitest <path>

Frequently Asked Questions

What test framework does openwork use?

The openwork repository uses Vitest as the underlying test runner, wrapped with a custom @openwork/testkit package for end-to-end browser automation. According to the source code in package.json and evals/runner/run.mjs, pnpm scripts coordinate test execution across the monorepo while the testkit provides page fixture and evidence recording capabilities.

Can I use npm or yarn instead of pnpm?

No. The codebase explicitly requires pnpm and defines workspace commands that depend on pnpm-specific features like --filter. Attempting to use npm or yarn will fail because the repository structure relies on pnpm workspaces and hoisting behavior configured in pnpm-workspace.yaml.

Where are the actual test files located?

End-to-end specs are in evals/specs/ with the .slow.test.ts suffix, while package-specific tests live in packages/[package-name]/tests/. The evals/specs/ directory contains the comprehensive evaluation suite that validates OpenWork features against real-running instances.

How do I run tests in a specific package only?

Use pnpm's filter flag to target a single workspace package. For example: pnpm --filter @openwork/app test or pnpm --filter @openwork/den-api test:admin-scale. The filter syntax ensures only that package's dependencies get installed and only its test configuration loads.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}