How the no-mistakes PR Summary Risk Assessment Algorithm Works (and Why It Ignores File Churn)

The no-mistakes PR summary risk assessment algorithm does not calculate risk from file churn or line counts; instead, it extracts structured RiskLevel and RiskRationale fields from JSON findings produced by the Review agent.

When no-mistakes generates a pull request summary, it appends a concise risk indicator like "✅ Low" or "⚠️ Medium" to communicate the potential impact of code changes. Contrary to common assumptions that such metrics derive from git diff statistics or file churn volumes, the no-mistakes PR summary risk assessment algorithm relies entirely on semantic analysis performed by the Review step. The algorithm is implemented in internal/pipeline/steps/prsummary.go and processes structured findings rather than counting added or removed lines.

How the Risk Assessment Algorithm Works

The core logic resides in the extractRiskLine function within internal/pipeline/steps/prsummary.go. The function iterates through pipeline step results to locate Review findings and converts them into a human-readable risk string.

Step 1: Locate the Review Step

The algorithm begins by scanning the slice of StepResult records to identify the entry where StepName equals StepReview. This is the only step considered for risk assessment according to the source code in prsummary.go.

Step 2: Parse Final Findings JSON

Once the Review step is located, the function attempts to unmarshal the FindingsJSON field into a types.Findings struct. If the JSON is malformed or unreadable, the function sets a flag but continues to evaluate fallback sources rather than failing immediately.

Step 3: Fallback to the Latest Round

If the final findings are absent or unreadable, the algorithm queries the rounds map using the step ID (sr.ID). It retrieves the most recent round from rounds[sr.ID] and attempts to parse the FindingsJSON from that round's data. This ensures that even if the final aggregation fails, the latest available assessment can still be used.

Step 4: Extract RiskLevel and RiskRationale

From the successfully parsed Findings object, the function extracts two critical fields:

• RiskLevel: A string enum value of "low", "medium", or "high"
• RiskRationale: Optional free-form text explaining the assessment

If RiskLevel is empty, the function returns an empty string and no risk line is rendered in the PR summary.

Step 5: Render the Risk Line with Emoji Mapping

The algorithm maps the risk level to a visual emoji using the riskEmoji helper function:

• ✅ "low" → Low
• ⚠️ "medium" → Medium
• 🚨 "high" → High
• ℹ️ Unknown values

The capitalizeRisk function formats the level as a capitalized label. If RiskRationale is present, it is appended after a colon. The final output follows this pattern:

✅ Low: straightforward refactor with no behavioural changes

Or without rationale:

⚠️ Medium

Why File Churn Is Not Used in Risk Calculation

Despite references to "churn" in other parts of the codebase (such as daemon comments about "churn the current run"), the PR summary risk assessment algorithm explicitly ignores file churn metrics like lines added, lines removed, or number of files modified. The assessment is purely deterministic based on the Review agent's structured output stored in types.Findings, as defined in internal/types/findings.go.

Implementation in the Source Code

The following Go implementation from internal/pipeline/steps/prsummary.go demonstrates the complete extraction logic:

// Inside internal/pipeline/steps/prsummary.go
func extractRiskLine(steps []*db.StepResult, rounds map[string][]*db.StepRound) string {
    for _, sr := range steps {
        if sr.StepName != types.StepReview { // 1️⃣ locate Review step
            continue
        }

        // 2️⃣ try to parse final findings
        var finalFindings *types.Findings
        hasUnreadable := false
        if sr.FindingsJSON != nil {
            if f, err := types.ParseFindingsJSON(*sr.FindingsJSON); err == nil {
                finalFindings = &f
            } else {
                hasUnreadable = true
            }
        }

        // 3️⃣ fallback to latest round if needed
        src := finalFindings
        if src == nil && !hasUnreadable {
            stepRounds := rounds[sr.ID]
            if len(stepRounds) > 0 {
                last := stepRounds[len(stepRounds)-1]
                if last.FindingsJSON != nil {
                    if f, err := types.ParseFindingsJSON(*last.FindingsJSON); err == nil {
                        src = &f
                    }
                }
            }
        }

        // 4️⃣ no risk information → empty line
        if src == nil || src.RiskLevel == "" {
            return ""
        }

        // 5️⃣ render emoji + capitalised label + optional rationale
        emoji := riskEmoji(src.RiskLevel)
        label := capitalizeRisk(src.RiskLevel)
        if src.RiskRationale != "" {
            return fmt.Sprintf("%s %s: %s", emoji, label, src.RiskRationale)
        }
        return fmt.Sprintf("%s %s", emoji, label)
    }
    return ""
}

Key Source Files

The risk assessment algorithm spans these four critical files in the kunchenguid/no-mistakes repository:

• internal/pipeline/steps/prsummary.go: Contains extractRiskLine, BuildPipelineSummary, and emoji mapping functions that convert findings into the rendered risk line.
• internal/types/findings.go: Defines the Findings struct with RiskLevel and RiskRationale fields.
• internal/pipeline/steps/pr.go: Orchestrates the PR description generation and inserts the risk line into the markdown body.
• internal/pipeline/steps/review.go: Implements the Review step that produces the findings JSON containing the risk assessment data.

Summary

• The no-mistakes PR summary risk assessment algorithm is implemented in extractRiskLine within internal/pipeline/steps/prsummary.go.
• Risk levels derive exclusively from the Review step's structured FindingsJSON, not from file churn or diff statistics.
• The algorithm supports a three-tier risk system (Low, Medium, High) with optional rationale text.
• A fallback mechanism ensures risk assessment persists even when final findings are corrupted by checking the latest round data.
• Output is formatted with emoji indicators (✅, ⚠️, 🚨) for immediate visual recognition in PR descriptions.

Frequently Asked Questions

Does the no-mistakes risk algorithm use file churn metrics?

No. The algorithm does not analyze file churn, line counts, or diff statistics when calculating risk. It relies solely on the RiskLevel and RiskRationale fields produced by the Review agent and stored in the findings JSON.

What happens if the Review step findings are unreadable?

If the final FindingsJSON cannot be parsed, the algorithm automatically falls back to the most recent round's findings for that step ID. If no parsable findings exist in either location, the function returns an empty string and no risk line appears in the PR summary.

Where is the risk line inserted in the PR description?

The BuildPipelineSummary function calls extractRiskLine and returns both the markdown body and the risk line. The PR command in internal/pipeline/steps/pr.go then inserts this risk line immediately after the generated Pipeline section in the pull request description.

What are the valid RiskLevel values?

According to the source code analysis, the algorithm expects RiskLevel to be one of three string values: "low", "medium", or "high". These map to ✅ Low, ⚠️ Medium, and 🚨 High respectively. Any other value receives the ℹ️ (unknown) emoji treatment.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}