Implementing Verifier Agents for Acceptance Criteria in Orchestrate

Implementing verifier agents for acceptance criteria in Orchestrate involves configuring specialized verification tasks that validate target worker tasks against predefined criteria, using Zod schemas for type safety and handoff parsing for deterministic state management.

Orchestrate, available in the cursor/plugins repository, structures complex workflows as executable plans containing three distinct task types: worker, subplanner, and verifier. A verifier agent acts as a gatekeeper that validates whether a specific target task satisfies its acceptance criteria before the workflow proceeds. This architecture ensures that quality checks remain explicit, traceable, and automated within the orchestration pipeline.

Understanding the Verifier Task Schema

The foundation of verifier implementation rests in orchestrate/skills/orchestrate/scripts/schemas.ts, where Zod schemas define the strict shape of verification tasks.

The Discriminated Union Pattern

The PlanTaskSchema discriminated union includes VerifierTaskSchema as a distinct variant requiring type: "verifier" and a mandatory verifies field. This field must contain a kebab-case ASCII string referencing the target task name to validate.

// orchestrate/skills/orchestrate/scripts/schemas.ts#L79-L87
const VerifierTaskSchema = z
  .object({
    ...planTaskBaseShape,
    type: z.literal("verifier"),
    verifies: z
      .string({ required_error: "is required" })
      .regex(TASK_NAME_RE, "must be kebab-case ascii")
      .describe("Name of the task this verifier checks."),
  })
  .strict();

Verification Result Enums

Verification outcomes use the VerificationSchema enum to maintain deterministic states across the system. Valid statuses include live-ui-verified, unit-test-verified, type-check-only, verifier-blocked, verifier-failed, and not-verified. These values populate the verification field on both the target task and verifier task rows for comprehensive auditing.

Generating Verifier Prompts with buildVerifierPrompt

When a verifier agent initializes, buildVerifierPrompt in orchestrate/skills/orchestrate/scripts/core/prompts.ts assembles a context-rich prompt that guides the verification process.

The function extracts the target task using the verifies reference, then injects critical context including the target's goal, acceptance checklist, verification plan, upstream handoff sections, and Slack metadata. This ensures the verifier agent operates with complete visibility into what constitutes successful completion.

// orchestrate/skills/orchestrate/scripts/core/prompts.ts#L51-L66
export function buildVerifierPrompt(
  t: Extract<PlanTask, { type: "verifier" }>,
  agentId: string | undefined,
  ctx: PromptRenderContext
): string {
    const target = planTasks(ctx.plan).find(x => x.name === t.verifies);
    …
    return renderPromptTemplate("verifier", {
      goal: ctx.plan.goal,
      scopedGoal,
      targetName,
      …
      ownVerifyPlan,
      upstream,
      slackBlock,
      startingRef: t.startingRef ?? targetBranch,
      branch,
    });
}

Parsing Verification Results from Handoffs

After execution, verifiers communicate results through handoff documents. The parseHandoffVerification function in orchestrate/skills/orchestrate/scripts/core/handoff.ts normalizes these outputs by reading the ## Verification section, with backward compatibility for legacy ## Verdict sections.

// orchestrate/skills/orchestrate/scripts/core/handoff.ts#L48-L71
export function parseHandoffVerification(handoff: string): Verification | null {
  const fromCanonical = readSectionValue({ handoff, heading: "Verification" });
  if (fromCanonical) {
    const normalized = normalizeEnumValue(fromCanonical);
    if (isVerification(normalized)) return normalized;
    return null;
  }
  const fromLegacy = readSectionValue({ handoff, heading: "Verdict" });
  if (!fromLegacy) return null;
  return mapLegacyVerdict(fromLegacy);
}

The function returns a validated Verification enum value or null if parsing fails, ensuring only recognized states enter the system.

State Propagation in AgentManager

The AgentManager class orchestrates verification state updates. When a verifier completes, the manager parses the handoff body and writes the verification result to both the target task and the verifier's own record using the touch method.

// orchestrate/skills/orchestrate/scripts/core/agent-manager.ts#L1095-L1105
const verification = parseHandoffVerification(handoffBody);
if (!verification) return;
if (target.verification === verification) return;
this.touch(target, { verification });
…
if (task.verification === verification) return;
this.touch(task, { verification });

This dual-write pattern maintains audit trails while preventing duplicate updates through idempotency checks.

Schema Validation for Verifies References

The PlanSchema enforces referential integrity at load time, ensuring every verifies field points to an existing task within the plan. Tests in orchestrate/skills/orchestrate/scripts/__tests__/schemas.test.ts confirm this validation prevents misconfigured plans.

// orchestrate/skills/orchestrate/scripts/__tests__/schemas.test.ts#L14-L22
test("PlanSchema accepts verifier that targets known task", () => {
  const plan = { …, tasks: [{ type: "verifier", verifies: "worker-1", … }] };
  expect(PlanObjectSchema.parse(plan)).toBeTruthy();
});

Practical Implementation Example

To implement a verifier agent, extend your plan JSON with a task definition that references an existing worker task via the verifies field.

{
  "name": "verify-frontend-toggle",
  "type": "verifier",
  "verifies": "worker-frontend-toggle",
  "scopedGoal": "Confirm UI toggle works end‑to‑end",
  "acceptance": [
    "UI displays toggle in header",
    "Toggle updates user preferences"
  ],
  "verify": "Run Cypress tests against the deployed preview"
}

When executed, the verifier receives a prompt containing the target's acceptance criteria and verification plan. Upon completion, the handoff body should include a verification section:

## Verification

verifier-failed

The AgentManager records this status on the target task, enabling downstream CI checks or dashboard indicators to surface acceptance criteria failures immediately.

Summary

• Verifier agents validate target tasks against acceptance criteria using specialized task definitions in Orchestrate workflows.
• The verifies field in VerifierTaskSchema establishes explicit dependencies using kebab-case task names validated by Zod schemas.
• Prompt generation via buildVerifierPrompt provides verifiers with comprehensive context including target goals and acceptance checklists.
• Handoff parsing through parseHandoffVerification normalizes verification results while maintaining backward compatibility with legacy formats.
• State propagation in AgentManager writes verification results to both target and verifier records, ensuring complete audit trails.

Frequently Asked Questions

What is a verifier agent in Orchestrate?

A verifier agent is a specialized task type in Orchestrate that validates whether another task (its target) satisfies predefined acceptance criteria. Unlike worker tasks that perform implementation work, verifiers act as quality gates that check outcomes against specifications defined in the plan's acceptance criteria.

How does verification state propagate between tasks?

When a verifier completes execution, AgentManager calls parseHandoffVerification to extract the result from the handoff document. It then uses the touch method to write the verification status to both the target task's verification field and the verifier's own record. This dual-write approach maintains referential integrity while creating comprehensive audit trails for debugging and compliance.

What are the valid verification statuses in Orchestrate?

The system recognizes six deterministic states defined in VerificationSchema: live-ui-verified for manual UI confirmation, unit-test-verified for automated test passage, type-check-only for static analysis completion, verifier-blocked when verification cannot proceed, verifier-failed when criteria are not met, and not-verified for pending states. These enums ensure consistent reporting across the orchestration pipeline.

How do I validate that my verifier configuration is correct?

Orchestrate validates verifier configurations at plan load time using Zod schema validation. The PlanSchema ensures that every verifies field references an existing task name within the plan. You can verify your configuration by running the test suite in orchestrate/skills/orchestrate/scripts/__tests__/schemas.test.ts, which confirms that verifiers target known tasks and conform to kebab-case naming conventions.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}