# OpenWork Performance Benchmarks: A Complete Technical Guide to Database, Workspace, and E2E Testing

> Explore OpenWork performance benchmarks for database, workspace, and E2E testing. This technical guide shows how to run benchmarks using pnpm for comprehensive performance analysis.

- Repository: [Different AI/openwork](https://github.com/different-ai/openwork)
- Tags: performance
- Published: 2026-08-04

---

**The different-ai/openwork repository provides three core performance benchmarks: an admin-scale MySQL database benchmark, a Windows workspace-session performance flow, and a hands-free computer-use e2e benchmark, all runnable via `pnpm` commands defined in the root [`package.json`](https://github.com/different-ai/openwork/blob/main/package.json).**

OpenWork is an open-source AI-powered workspace platform that ships with comprehensive performance validation to ensure responsiveness across its database layer, Electron desktop client, and agent-driven workflows. This guide explains how each benchmark works architecturally, where the source code lives, and how to run them yourself.

## Admin-Scale MySQL Benchmark

The most heavily instrumented benchmark measures end-to-end latency of the admin-panel database layer using a disposable MySQL instance.

### What It Measures

This benchmark in [[`ee/packages/den-db/scripts/admin-scale-benchmark.ts`](https://github.com/different-ai/openwork/blob/main/ee/packages/den-db/scripts/admin-scale-benchmark.ts)](https://github.com/different-ai/openwork/blob/dev/ee/packages/den-db/scripts/admin-scale-benchmark.ts) captures:

- **Initial connection latency** to MySQL
- **User search execution time**
- **Organization search execution time**

The script automatically provisions a temporary database at `mysql://root:password@127.0.0.1:3306/openwork_admin_scale_benchmark` and destroys it after completion.

### Architecture and Safety Guards

The benchmark enforces clean-state execution. From the source:

- It checks for `git status --porcelain` and throws if uncommitted changes exist
- Uses `tsx` for TypeScript execution without separate compilation
- Writes structured JSON results to [`evals/results/admin-scale-performance-benchmark/latest.json`](https://github.com/different-ai/openwork/blob/main/evals/results/admin-scale-performance-benchmark/latest.json)

### How to Run

```bash

# Install dependencies first

pnpm install

# Execute the MySQL admin-scale benchmark

pnpm benchmark:admin-scale:mysql

```

Sample output includes timing breakdowns:

```

Resetting disposable benchmark database admin_scale_benchmark
... 500 ms initial, 300 ms searches

```

## Windows Workspace-Session Performance Flow

This benchmark validates Electron desktop performance under heavy workspace and session load.

### What It Measures

Located in [`evals/flows/windows-workspace-session-performance.flow.mjs`](https://github.com/different-ai/openwork/blob/dev/evals/flows/windows-workspace-session-performance.flow.mjs), this benchmark tests:

- **Workspace startup time** on Windows
- **Bulk session creation** (default: 20 workspaces, 200 total sessions)
- **UI navigation latency** between sessions
- **Responsive threshold compliance**

The accompanying narrative document at [[`evals/voiceovers/windows-workspace-session-performance.md`](https://github.com/different-ai/openwork/blob/main/evals/voiceovers/windows-workspace-session-performance.md)](https://github.com/different-ai/openwork/blob/dev/evals/voiceovers/windows-workspace-session-performance.md) describes the testing intent and pass criteria.

### Test-Kit Integration

The flow uses the **@openwork/testkit** framework with `ctx.prove` and `ctx.expectText` APIs. It captures:

- Screenshots as evidence (tagged "admin-scale-budget-evidence")
- Timing assertions with `benchmarkSessionCount` and `benchmarkSessionTotal` variables
- Automatic profile isolation via `OPENWORK_DEV_PROFILE=auto`

### How to Run

```bash

# Start the dev build

pnpm dev

# Run the workspace-session performance flow

pnpm test:e2e windows-workspace-session-performance

```

Runtime logs show progress:

```

benchmark session ready <sessionId>
benchmarkSessionCount: 20
benchmarkSessionTotal: 200

```

## Hands-Free Computer-Use E2E Benchmark

The third benchmark measures end-to-end quality and latency of the autonomous "hands-free" agent flow.

### Platform-Specific Behavior

Found in [`packages/handsfree/test/e2e/run.mjs`](https://github.com/different-ai/openwork/blob/dev/packages/handsfree/test/e2e/run.mjs), this benchmark:

- **Skips entirely on non-macOS platforms** with message: *"computer-use e2e benchmark is macOS-only; skipping"*
- **Executes full agent loop** on macOS, measuring perception-to-action latency
- **Validates hands-free quality metrics** including task completion rate and response time

### How to Run

```bash

# Run as part of standard e2e suite

pnpm test:e2e handsfree

```

Or execute all e2e benchmarks together:

```bash
pnpm test:e2e

```

## Benchmark Infrastructure and CI Integration

All benchmarks share common infrastructure patterns according to the OpenWork source code:

| Pattern | Implementation |
|--------|----------------|
| **Disposable resources** | Temporary MySQL containers, fresh Electron profiles |
| **CLI entry points** | [`package.json`](https://github.com/different-ai/openwork/blob/main/package.json) scripts like `benchmark:admin-scale:mysql` |
| **Result persistence** | JSON files in `evals/results/` directories |
| **Evidence capture** | Screenshots and timing logs for regression analysis |

The root [[`package.json`](https://github.com/different-ai/openwork/blob/main/package.json)](https://github.com/different-ai/openwork/blob/dev/package.json) defines the benchmark orchestration layer, using `node --import tsx` for TypeScript execution without build steps.

## Summary

- **Admin-scale MySQL benchmark** validates database layer performance with disposable instances and structured JSON reporting.

- **Windows workspace-session flow** tests Electron desktop scalability under 200+ session loads using the testkit framework.

- **Hands-free e2e benchmark** measures autonomous agent quality, macOS-only, integrated into standard test suites.

- All benchmarks enforce clean Git state, isolate resources, and output machine-readable results for CI tracking.

## Frequently Asked Questions

### How do I add a new performance benchmark to OpenWork?

Create a `.flow.mjs` file in `evals/flows/` following the testkit pattern used in [`windows-workspace-session-performance.flow.mjs`](https://github.com/different-ai/openwork/blob/dev/evals/flows/windows-workspace-session-performance.flow.mjs). Add a narrative description in `evals/voiceovers/` and register a `pnpm` script in root [`package.json`](https://github.com/different-ai/openwork/blob/main/package.json) for discoverability.

### Why does the hands-free benchmark skip on my machine?

The intentional platform restriction in [`packages/handsfree/test/e2e/run.mjs`](https://github.com/different-ai/openwork/blob/dev/packages/handsfree/test/e2e/run.mjs) limits execution to macOS because the underlying computer-use APIs depend on macOS accessibility frameworks. This is logged explicitly when running on Linux or Windows.

### Where are benchmark results stored?

JSON artifacts persist to `evals/results/<benchmark-name>/latest.json` at runtime. The admin-scale benchmark generates this path automatically; other flows may configure output locations via environment variables in their testkit configuration.

### Can I run benchmarks without installing MySQL locally?

Yes. The admin-scale benchmark in [[`ee/packages/den-db/scripts/admin-scale-benchmark.ts`](https://github.com/different-ai/openwork/blob/main/ee/packages/den-db/scripts/admin-scale-benchmark.ts)](https://github.com/different-ai/openwork/blob/dev/ee/packages/den-db/scripts/admin-scale-benchmark.ts) expects a running MySQL instance at the documented connection string, but you can modify the connection parameters or use containerized MySQL via Docker for isolated testing.