How the Review-Fix Verification Contract Operates in the No-Mistakes Pipeline

The review-fix verification contract is a set of rules encoded in ReviewStep.Execute that requires the fixer to apply all intended changes before running a single focused verification, while strictly prohibiting full test or lint suite execution during the fix round.

The review-fix verification contract governs how automated fixes are validated within the no-mistakes repository's review step. This contract ensures that fix operations remain efficient while maintaining correctness through strategic deferral of comprehensive verification. By encoding specific constraints directly into the pipeline's prompt generation logic in internal/pipeline/steps/review.go, the system prevents the 5× performance degradation observed in early measurements while ensuring fixes are properly validated later.

Contract Activation in Fix Mode

The contract activates when the pipeline runs in fix mode, detected via sctx.Fixing == true within the ReviewStep.Execute implementation. When this condition is met, the step constructs a fixPrompt (lines 72‑83) that embeds the contract rules directly into the instructions sent to the agent. This is not a sandbox restriction—the agent retains full shell access—but rather a performance and workflow discipline enforced through prompt engineering and regression testing.

The Three Core Contract Rules

The contract specified in internal/pipeline/steps/review.go (lines 78‑81) consists of three strict clauses that govern the fixer's behavior:

Apply All Fixes Before Any Verification

The fixer must make all intended changes before running verification. According to lines 78‑80, the prompt explicitly states: "Apply all the fixes you intend to make first; do not run any verification in between individual fixes." This batching prevents the overhead of repeated verification cycles and ensures the agent focuses on implementation before validation.

Single Focused Verification Only

After all fixes are applied, the fixer runs one verification limited to the changed area. As specified at lines 79‑80, the contract requires: "After all fixes are applied, run one focused verification limited to the changed area … at the end of the fix round to confirm the fixes hold." This targets specific packages, files, or tests rather than re-examining the entire repository.

Prohibition of Full Test and Lint Suites

The contract explicitly forbids running the complete repository test or lint suite during the fix round. Lines 80‑81 state: "DO NOT run the complete repository test suite or lint suite during this fix round. The pipeline has dedicated test and lint steps after review." This deferral is critical because the pipeline order defined in internal/pipeline/common.go places Review before Test and Lint, ensuring comprehensive validation occurs later without duplication.

Performance Rationale

The contract exists primarily for wall-clock efficiency. According to the comment block at lines 40‑47 in review.go, running the full verification suite during every fix round caused a 5× slowdown in measurements, consuming approximately 784 seconds of a 2,419 second review step. By limiting verification to a single focused check and deferring comprehensive testing to dedicated steps, the pipeline maintains speed while still guaranteeing correctness through the subsequent Test and Lint steps.

Implementation in ReviewStep.Execute

The contract is implemented through three distinct phases in internal/pipeline/steps/review.go:

1. Building the Contract into the Fix Prompt

Lines 72‑83 construct the fixPrompt by embedding the contract rules directly into the agent's instructions:

fixPrompt := fmt.Sprintf(
    `...  
    Rules:
    - Apply all the fixes you intend to make first; do not run any verification in between individual fixes.
    - After all fixes are applied, run one focused verification limited to the changed area …
    - DO NOT run the complete repository test suite or lint suite during this fix round …
    ...`,
    // other interpolated variables omitted for brevity
)

2. Executing Fix Mode with Contract Constraints

The prompt is passed to executeFixMode at lines 96‑105 along with execution options that enforce the contract context:

summary, err := executeFixMode(sctx, s.Name(), fixExecutionOptions{
    RequirePreviousFindings: true,
    LogMessage:              "asking agent to fix identified issues...",
    Prompt:                  fixPrompt,
    SessionRole:             pipeline.SessionRoleFixer,
    Purpose:                 "review-fix",
    Workload:                workload,
})

3. Returning the Fix Summary

After execution, the step returns a FixSummary (lines 52‑54) that captures the results of the fix round without the full verification overhead:

return &pipeline.StepOutcome{
    Findings:   string(findingsJSON),
    FixSummary: fixSummary,
}

Enforcement Through Regression Testing

Because the agent retains full shell access, the contract is enforced not through sandboxing but through regression tests that validate the prompt wording. The test TestReviewStep_FixMode_FocusedVerificationContract in internal/pipeline/steps/review_test.go checks that the contract text remains present in the generated prompt. If the prompt deviates from the specified rules, the test fails, ensuring the contract stays synchronized with the implementation.

Summary

• The review-fix verification contract activates when sctx.Fixing == true in ReviewStep.Execute.
• It mandates three strict rules: apply all fixes before verification, run only one focused verification, and never execute the full test or lint suite during the fix round.
• This design recovered 784 seconds of execution time by eliminating redundant verification, achieving a 5× speedup over the naive approach.
• Enforcement occurs via regression tests on prompt text rather than sandbox restrictions, ensuring the contract remains intact as the codebase evolves.
• Comprehensive validation is deferred to the dedicated Test and Lint steps that follow the review step in the pipeline.

Frequently Asked Questions

When does the review-fix verification contract activate?

The contract activates when the pipeline step detects fix mode via the condition sctx.Fixing == true in internal/pipeline/steps/review.go. This boolean indicates that the review step should operate as a fixer rather than a reviewer, triggering the construction of the fixPrompt containing the contract rules.

Why can't the fixer run the full test suite during the review step?

Running the complete repository test or lint suite during the fix round would duplicate work and severely impact performance. According to measurements in lines 40‑47 of review.go, this approach previously consumed 784 seconds of a 2,419 second review step. The pipeline architecture delegates comprehensive validation to dedicated Test and Lint steps that execute after the review step completes.

How is the contract enforced if the agent has full shell access?

The contract is enforced through regression testing rather than sandboxing. The test TestReviewStep_FixMode_FocusedVerificationContract validates that the contract wording appears in the generated prompt. If a developer modifies the prompt text and removes the contract rules, the test fails immediately, preventing accidental contract violations.

What happens after the fix summary is returned?

After executeFixMode returns the FixSummary, the ReviewStep stores this summary in the StepOutcome (lines 52‑54). The pipeline later uses this summary to create a commit message, while the subsequent Test and Lint steps run the full verification suite to guarantee that the fixes pass all repository checks.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}