Fraimz Validation Framework: How to Create Frame-by-Frame Proof in OpenWork

The fraimz validation framework is OpenWork's canonical proof system for validating that any change—whether a feature, bug‑fix, refactor, or internal update—behaves as intended in the real application by generating a single, inspectable HTML artifact.

This article explains how the fraimz framework works, where it lives in the codebase, and how to create frame‑by‑frame proof that your changes work correctly.

What Is the Fraimz Framework?

A fraimz is a single HTML artifact located at evals/results/<run‑id>/fraimz.html that contains a sequence of frames. Each frame binds together five elements that prove a specific user‑visible outcome:

• Claim — A one‑line description of the expected outcome (e.g., "user can create a task")
• Action — The interaction performed (click, type, API call, etc.)
• Assertion — The observable side‑effect that proves the claim (DOM text, HTTP response, file, DB row)
• Voiceover — A short narration that serves as the specification for the frame
• Screenshot — A validated image or terminal output showing the UI state

The framework enforces a demo‑first workflow: you write the voice‑over script before writing any code, then scaffold and drive the flow against the real application via Chrome DevTools Protocol (CDP).

The 7-Step Fraimz Workflow

According to the fraimz skill, the loop follows these stages:

1. Frame the experience — Write a concise claim and choose a demo kind (user‑facing or internal)
2. Get the script — Approve a voice‑over in evals/voiceovers/<flow‑id>.md (owned by the voiceover skill)
3. Scaffold the flow — Run pnpm fraimz scaffold <flow‑id> to generate a flow file in evals/flows/
4. Code the flow — Use ctx.* helpers (ctx.click, ctx.fill, ctx.assert, ctx.screenshot) to drive the app
5. Drive it for real — Launch the app and execute: pnpm fraimz --flow <flow‑id> --cdp-url <electron‑cdp‑url>
6. Validate & repair — Edit and re‑run until every frame passes
7. Verdict — The runner writes fraimz.html, report.md, and report.json

A run is marked Passed only when the HTML exists and every claim is backed by a passing assertion; otherwise it is Incomplete or Failed.

How to Create Frame-by-Frame Proof

Step 1: Scaffold from Voice‑Over

Start with an approved voice‑over script, then generate the flow skeleton:

# Creates evals/flows/my-new-flow.flow.mjs from evals/voiceovers/my-new-flow.md

pnpm fraimz scaffold my-new-flow

Step 2: Code the Flow with ctx Helpers

Inside your .flow.mjs file, use the ctx.* API to interact with the UI via CDP. Each frame calls ctx.prove() with a claim, voice‑over, action, assertion, and screenshot:

// evals/flows/my-new-flow.flow.mjs
export default async (ctx) => {
  // Frame 1 – claim + voice‑over
  await ctx.prove("user can create a new task", {
    voiceover: "Click the + button, type a title, and press Enter.",
    action: async () => {
      await ctx.clickText("+")
      await ctx.fill("Task title", "Demo task")
      await ctx.pressEnter()
    },
    assert: async () => {
      await ctx.expectText("Demo task")
    },
    screenshot: async (name) => {
      await ctx.screenshot(name, {
        claim: "task appears in the list",
        voiceover: "The new task is now listed below.",
        requireText: ["Demo task"],
      })
    },
  })
}

The ctx.* helpers are documented in [evals/README.md](https://github.com/different-ai/openwork/blob/dev/evals/README.md) and include methods for clicks, fills, navigation, assertions, and screenshot capture with validation rules.

Step 3: Execute Against a Real App Instance

Start OpenWork with CDP exposed, then run your flow:

# Terminal 1: Start OpenWork in Electron with remote debugging

OPENWORK_ELECTRON_REMOTE_DEBUG_PORT=9826 pnpm dev

# Terminal 2: Run the fraimz against the live CDP endpoint

pnpm fraimz --flow my-new-flow --cdp-url http://127.0.0.1:9826

Step 4: Publish Proof to Your PR

Once validation passes, post the generated proof as a PR comment:

# Posts fraimz.html as a comment on the current PR

pnpm fraimz --flow my-new-flow --pr

# Or specify a PR number explicitly

pnpm fraimz --flow my-new-flow --pr 42

Key Architectural Components

Component	Location	Purpose
Skill definition	.opencode/skills/fraimz/SKILL.md	Full loop definition, demo kinds, required helpers
Command wrapper	.opencode/commands/fraimz.md	CLI documentation (pnpm fraimz …) and options
Eval runner	evals/README.md	Orchestration, voice‑over parsing, frame validation
Flow API	evals/flows/*.flow.mjs	ctx.* helpers for CDP‑driven UI interaction
Voice‑over scripts	evals/voiceovers/*.md	Source‑of‑truth specifications for each frame

The framework is invoked automatically when a task ends with "please create a fraimz" or when any change might affect observable behavior, as specified in AGENTS.md under the "fraimz" section.

Summary

• The fraimz validation framework produces a single fraimz.html artifact that proves changes work in the real application
• Each frame combines claim, action, assertion, voiceover, and screenshot into verifiable proof
• The demo‑first workflow requires voice‑over approval before code implementation
• Use pnpm fraimz scaffold <flow‑id> to generate flows, ctx.* helpers to code interactions, and pnpm fraimz --flow <id> --cdp-url <url> to execute
• A run passes only when fraimz.html exists and every assertion succeeds

Frequently Asked Questions

What makes a fraimz different from traditional unit tests?

A fraimz validates behavior through the real UI using Chrome DevTools Protocol, producing a human‑readable HTML artifact with screenshots and narrated steps. Unlike unit tests that mock dependencies, a fraimz proves the actual user experience works as specified.

Where are voice‑over scripts stored and who owns them?

Voice‑over scripts live at evals/voiceovers/<flow‑id>.md and are owned by the voiceover skill. They must be approved before scaffolding can begin, ensuring the specification is clear before implementation.

Can fraimz runs be automated in CI?

Yes. The framework supports headless execution via CDP endpoints. Pass --cdp-url to connect to any Chrome or Electron instance, including containerized or sandboxed environments like Daytona.

What happens if a screenshot or assertion fails?

The runner marks the frame as failed, writes diagnostic output to report.md and report.json, and exits with a non‑zero status. You repair the flow—updating selectors, timing, or logic—and re‑run until all frames pass validation.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}