How to Test OpenWork Components: A Complete Guide to @openwork/testkit

Test OpenWork components using @openwork/testkit specs located in evals/specs/**/*.test.ts, recording observable assertions on an ambient evidence tape that can be published for CI verification.

OpenWork uses a specialized end-to-end testing framework called @openwork/testkit to verify UI components across Electron, Den, and web dashboard surfaces. The testing workflow isolates component verification in the evals/ workspace, producing durable evidence tapes that serve as PR-verified proof of correct behavior.

Understanding the OpenWork Testing Architecture

The testing system follows a three-stage workflow designed to bridge local development with CI verification. Each spec is a self-contained TypeScript file that declares dependencies, exercises the UI, and records persistent claims.

The Three-Stage Workflow

Stage	Purpose	Core API
Author	Write a spec describing the user journey and asserting observable outcomes	import { test, needs, server, app } from "@openwork/testkit"
Run	Execute locally or in a Daytona sandbox with auto-provisioned dependencies	pnpm --dir evals exec vitest run --project stack specs/<slug>.slow.test.ts
Publish	Convert the recorded tape into a PR-verified evidence bundle	pnpm evals --publish --pr <pr-number>

Core Testkit APIs

The @openwork/testkit package exports several critical functions for component testing:

• test() – Defines a spec that receives an evidence object for recording persistent claims
• needs() – Declares external services (MySQL, mock MCP, OAuth); skips the test with a meaningful reason if unavailable
• server() – Spins up a local HTTP server serving the app under test
• app() – Launches the Electron/Den/Web surface and returns a CDP client for low-level interaction

All assertions that must survive beyond the test run require evidence.fact(). This method writes to the evidence tape that CI verifiers check.

Writing Your First Component Spec

Specs follow a consistent pattern combining Vitest assertions with OpenWork's evidence recording system.

Spec Structure and Pattern

A valid spec imports from @openwork/testkit and uses the standard Arrange-Act-Assert structure:

import { test } from "@openwork/testkit";

test("description of the behavior", async ({ evidence }) => {
  // 1️⃣ Arrange – load files, call helpers, etc.
  // 2️⃣ Act – invoke the UI, e.g. click a button, fill a form
  // 3️⃣ Assert – use Vitest’s `expect` and `evidence.fact` for durable claims
});

Component tests that drive an app surface must use the .slow.test.ts suffix. Files using only .test.ts run as unit-only tests and lack the required sandbox resources.

Recording Evidence with evidence.fact

The evidence.fact() method creates observable claims that appear in the final evidence tape. Without this call, a passing test is marked Incomplete for PR verification purposes.

Consider this real-world example from evals/specs/welcome-one-field.slow.test.ts, which tests the welcome screen's input parsing:

import { readFileSync } from "node:fs";
import { fileURLToPath } from "node:url";
import { expect } from "vitest";
import { test } from "@openwork/testkit";
import {
  parseInviteLinkInput,
  parseServerUrlInput,
} from "../../apps/app/src/react-app/domains/cloud/join-organization-input";

const welcomePagePath = fileURLToPath(
  new URL("../../apps/app/src/react-app/domains/onboarding/welcome-page.tsx", import.meta.url)
);
const dialogPath = fileURLToPath(
  new URL("../../apps/app/src/react-app/domains/cloud/join-organization-dialog.tsx", import.meta.url)
);
const enLocalePath = fileURLToPath(
  new URL("../../apps/app/src/i18n/locales/en.ts", import.meta.url)
);

test("the welcome screen folds the server‑URL door into the join‑organization field", async ({ evidence }) => {
  const welcomeSource = readFileSync(welcomePagePath, "utf8");
  const dialogSource = readFileSync(dialogPath, "utf8");
  const enLocale = readFileSync(enLocalePath, "utf8");

  // Parsing helpers validation
  const invite = parseInviteLinkInput("https://den.acme.test/join-org?invite=inv_123");
  expect(invite).toEqual({
    url: "https://den.acme.test/join-org?invite=inv_123",
    origin: "https://den.acme.test",
    host: "den.acme.test",
  });
  expect(parseInviteLinkInput("https://den.acme.test/install?token=abc")).toBeNull();
  expect(parseServerUrlInput("https://openwork.acme.test/")).toEqual({
    url: "https://openwork.acme.test",
    host: "openwork.acme.test",
  });

  // UI source checks
  expect(welcomeSource).not.toContain("OrganizationServerAffordance");
  expect(welcomeSource).toContain('data-testid="welcome-join-org"');
  expect(enLocale).toContain('"welcome.join_org_subtitle": "Paste your invite link, install link, or server URL"');

  expect(dialogSource).toContain("if (await submitInstallLink(trimmedInput)) return;");
  expect(dialogSource).toContain("if (await submitInviteLink(trimmedInput)) return;");
  expect(dialogSource).toContain("if (await submitServerUrl(trimmedInput)) return;");

  // Persistent claim for evidence tape
  evidence.fact(
    "The welcome screen has one paste field for invite links, install links, server URLs, and sign‑in codes",
    "The separate Using OpenWork on‑premises affordance is gone from Welcome; the join dialog classifies install link, then invite link, then server URL, then sign‑in code, and requires explicit host confirmation before opening web invites.",
    true,
  );
});

Source: welcome-one-field.test.ts

This test validates both the parsing logic in apps/app/src/react-app/domains/cloud/join-organization-input and the UI structure in apps/app/src/react-app/domains/onboarding/welcome-page.tsx.

Running Tests Locally and in CI

The evals/ workspace operates independently from the main product installation, requiring separate dependency management.

Local Execution

First, install the isolated workspace dependencies:

pnpm --dir evals install

Execute a single spec using Vitest with the stack project configuration:

OPENWORK_EVAL_APP_SPECS=1 \
  pnpm --dir evals exec vitest run --project stack specs/welcome-one-field.slow.test.ts

The OPENWORK_EVAL_APP_SPECS=1 environment flag is mandatory for tests that drive the App surface. Without it, the test runner lacks the infrastructure to launch Electron or web contexts.

Daytona Sandbox Testing

For CI-compatible execution or isolated environments, run tests inside a Daytona sandbox:

OPENWORK_EVAL_DAYTONA=1 \
  OPENWORK_EVAL_APP_SPECS=1 \
  pnpm --dir evals exec vitest run --project stack specs/<slug>.slow.test.ts

Setting OPENWORK_EVAL_DAYTONA=1 automatically provisions declared dependencies via the needs() API within the sandbox environment.

Publishing Evidence Tapes

Once a spec passes, publish its evidence tape to attach verifiable proof to a pull request:

pnpm evals --publish --pr 1234 [--roll <dir|name>]

Only tapes containing observable claims via evidence.fact() can achieve Passed status. Tests using only Vitest's expect without recording facts are marked Incomplete and cannot satisfy PR verification requirements.

The CLI also supports bulk execution (pnpm evals <spec-names…>) and vision-enabled validation (--with-llm-vision) for AI-assisted assertions.

Common Pitfalls to Avoid

When you test OpenWork components, avoid these frequent mistakes:

• Missing evidence.fact() – The test may pass locally but will be treated as Incomplete in CI because no durable evidence exists for verification.
• Undeclared dependencies – Always use needs() for external services like MySQL or OAuth; otherwise, the spec fails with nondeterministic errors when those services are unavailable.
• Incorrect file suffix – App-driving specs must use .slow.test.ts. Using .test.ts causes the test to run as a unit test without the required sandbox resources, leading to failures when calling app() or server().

Summary

• Test OpenWork components using the @openwork/testkit framework in the isolated evals/ workspace
• Write specs in evals/specs/**/*.slow.test.ts using the test() function and evidence.fact() for persistent claims
• Import testing utilities from @openwork/testkit: test, needs, server, and app
• Run locally with OPENWORK_EVAL_APP_SPECS=1 and the Vitest stack project
• Publish evidence tapes using pnpm evals --publish --pr <number> for CI verification
• Consult evals/README.md and evals/specs/welcome-one-field.slow.test.ts for implementation reference

Frequently Asked Questions

What is the difference between .test.ts and .slow.test.ts files in OpenWork?

Files ending in .slow.test.ts are recognized by the test runner as end-to-end specs that require sandbox resources like Electron instances or HTTP servers. Files using only .test.ts run as fast unit tests without access to app(), server(), or the full environment provisioning system. Always use .slow.test.ts when testing components that interact with the UI surface.

How does evidence.fact() differ from Vitest's expect()?

expect() assertions verify behavior during the test run but disappear once the process exits. evidence.fact() records persistent claims to an evidence tape that survives the test execution. According to the OpenWork source code, only specs containing evidence.fact() calls can be marked Passed for PR verification; tests using only expect() are classified as Incomplete regardless of whether they pass locally.

Can I run OpenWork component tests without installing the main application?

Yes. The evals/ workspace is intentionally isolated from the main product install. You only need to run pnpm --dir evals install to set up the testing environment. The testkit handles provisioning dependencies declared via needs(), allowing you to test OpenWork components without configuring the full production stack locally.

What happens if I forget to declare a dependency with needs()?

If your test requires an external service (MySQL, mock MCP server, OAuth provider) but you don't declare it with needs(), the test will likely fail with a nondeterministic error when that service is unavailable. The needs() API ensures the test is skipped with a meaningful reason rather than failing, making the test suite more robust across different environments.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}