How to Run OpenWork e2e Tests and Fraimz Validation in Development

Run pnpm test:e2e for the standard e2e suite and pnpm fraimz --flow <id> --cdp-url <url> to generate a fraimz proof artifact against a local or sandboxed Electron instance.

OpenWork provides a unified test harness for verifying both automated end-to-end tests and narrative-driven fraimz validation artifacts. This guide covers the exact commands, configuration, and source locations you need to run both test types during local development.

Run the Standard e2e Test Suite

The root package.json defines the test:e2e script in dev/package.json at lines 55-57:

"test:e2e": "pnpm --filter @openwork/app test:e2e"

This delegates execution to the @openwork/app workspace, where the actual test runner lives in apps/app/src/react-app/tests/e2e/ orchestrated by scripts/e2e.mjs.

Execute the full suite from repository root:

pnpm test:e2e

The command automatically starts a dev Electron instance and drives it via Chrome DevTools Protocol (CDP), exercising health checks, session handling, and event propagation flows.

Filter to specific tests:

pnpm --filter @openwork/app test:e2e --grep "session"

Generate a Fraimz Proof Artifact

Fraimz transforms verified UI flows into self-contained HTML reports (fraimz.html). The entry point is the fraimz script defined in evals/runner/run.mjs:

"fraimz": "node evals/runner/run.mjs --mode demo"

Prerequisites

1. Voice-over script at evals/voiceovers/<flow-id>.md — describes the user story in natural language
2. Flow scaffold at evals/flows/<flow-id>.flow.mjs — generated executable test

Create and Run a Fraimz Flow

Step 1: Scaffold a new flow

pnpm fraimz scaffold my-feature

This creates evals/flows/my-feature.flow.mjs from your approved voice-over.

Step 2: Start the dev Electron instance with CDP enabled

OPENWORK_ELECTRON_REMOTE_DEBUG_PORT=9826 pnpm dev &

Step 3: Execute the fraimz against the local CDP endpoint

pnpm fraimz --flow my-feature --cdp-url http://127.0.0.1:9826

The runner captures screenshots, asserts UI state at each step, and outputs:

• evals/results/<run-id>/fraimz.html — human-readable report
• evals/results/<run-id>/report.json — machine-readable results

Per the fraimz skill specification in .opencode/skills/fraimz/SKILL.md, the loop continues until all frames pass or you fix the underlying code and re-run.

Run Fraimz Validation in a Daytona Sandbox

For fully isolated testing, use the daytona-electron-test skill:

bash .devcontainer/test-on-daytona.sh my-branch --artifacts-volume

The script prints a CDP URL (e.g., http://127.0.0.1:9825). Pass this to fraimz:

pnpm fraimz --flow my-feature --cdp-url http://127.0.0.1:9825

Complete Development Workflow

A typical iteration combining both test types:

# One-time setup

pnpm install

# Fast feedback

pnpm typecheck
pnpm test:e2e

# Full narrative validation

OPENWORK_ELECTRON_REMOTE_DEBUG_PORT=9826 pnpm dev &
pnpm fraimz --flow new-task-creation --cdp-url http://127.0.0.1:9826

Re-run fraimz until fraimz.html shows Passed for all frames.

Key Source Files

Path	Purpose
dev/package.json	Root test:e2e and fraimz script definitions
apps/app/package.json	Workspace implementing the e2e runner
evals/runner/run.mjs	Core fraimz/evals runner entry point
.opencode/skills/fraimz/SKILL.md	Fraimz loop specification and output format
evals/flows/	Reusable flow implementations (e.g., session-search-grouped.flow.mjs)
evals/voiceovers/	Approved narration scripts
apps/app/src/react-app/ARCHITECTURE.md	E2e smoke test architecture notes

Summary

• Standard e2e: Run pnpm test:e2e to verify core flows via CDP-driven Electron automation
• Fraimz validation: Scaffold with pnpm fraimz scaffold, then execute with --flow and --cdp-url against a local dev instance or Daytona sandbox
• CDP requirement: Both test types require OPENWORK_ELECTRON_REMOTE_DEBUG_PORT set to expose the Chrome DevTools Protocol endpoint
• Output artifacts: Fraimz generates fraimz.html and report.json in evals/results/<run-id>/

Frequently Asked Questions

What is the difference between pnpm test:e2e and pnpm fraimz?

pnpm test:e2e runs traditional automated tests located in apps/app/src/react-app/tests/e2e/ — fast, deterministic checks of health, sessions, and events. pnpm fraimz executes narrative-driven flows defined by voice-overs and produces human-readable HTML proof artifacts suitable for design review or compliance documentation.

Why does my fraimz fail with "CDP connection refused"?

The Electron dev instance must launch with OPENWORK_ELECTRON_REMOTE_DEBUG_PORT set. Without this environment variable, the CDP endpoint is unavailable. Verify the port matches your --cdp-url argument and that no other process occupies it.

Can I run fraimz without writing a voice-over?

No. Per .opencode/skills/fraimz/SKILL.md, the fraimz loop requires an approved voice-over at evals/voiceovers/<flow-id>.md as the source of truth for narration and step sequencing. Run pnpm fraimz scaffold <flow-id> after creating the markdown file.

How do I debug a failing fraimz frame?

Inspect evals/results/<run-id>/fraimz.html for the failing screenshot and assertion message. Fix the underlying code or adjust the flow definition in evals/flows/<flow-id>.flow.mjs, then re-run the fraimz command against the same CDP URL.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}