How to Configure Risk-Scored PR Reviews in GitHub Actions

The code-review-graph repository provides a composite GitHub Action that analyzes pull requests locally, assigns risk scores (0-1) to changed symbols, posts a sticky comment with results, and optionally fails the workflow when risk thresholds are exceeded.

You can implement risk-scored PR reviews in your CI pipeline using the composite action from tirth8205/code-review-graph. This tool builds a dependency graph of your codebase, calculates risk factors for every changed function or class in a pull request, and surfaces the results directly in GitHub's PR interface. The action runs entirely on the GitHub Actions runner, requires no external services, and supports caching for incremental analysis.

Understanding the Composite Action Architecture

The solution centers on a composite action defined in action.yml at the repository root. This action orchestrates nine distinct steps that handle environment setup, graph construction, risk analysis, and result reporting.

The workflow installs Python 3.12 by default, caches the SQLite graph database in .code-review-graph/ using a schema-versioned key (schema9), and determines the base reference for comparison using github.base_ref for PR events or HEAD~1 for other triggers. When a cache hit occurs, the action runs code-review-graph update for incremental builds; otherwise, it performs a full build operation.

The analysis phase executes code-review-graph detect-changes --base "$CRG_BASE" to generate a JSON report containing per-symbol risk scores and an overall aggregate. The scripts/render_pr_comment.py script then transforms this JSON into a markdown table, embeds a hidden HTML marker (<!-- code-review-graph-report -->), and writes the output to crg-comment.md.

How Risk Scoring Works

The risk score is a floating-point value between 0 and 1 calculated from multiple heuristics including flow participation, community crossing (dependencies between modules), test coverage gaps, security-sensitive naming patterns, and caller count. Higher scores indicate changes that touch critical or complex parts of the system.

The action maps these scores to four distinct risk levels as documented in docs/GITHUB_ACTION.md (lines 78-89):

Level	Score Range
low	< 0.40
medium	0.40 – 0.69
high	0.70 – 0.84
critical	≥ 0.85

You can configure the fail-on-risk input to enforce quality gates. When set to high or critical, the action invokes render_pr_comment.py with the --fail-on-risk flag, which exits with status code 3 if the overall risk exceeds the threshold, causing the GitHub Actions job to fail and block merging.

Step-by-Step Configuration Guide

Minimal Setup

Create a workflow file at .github/workflows/code-review-graph.yml with the following structure:

name: code-review-graph
on:
  pull_request:
permissions:
  contents: read
  pull-requests: write
jobs:
  review:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v7
      - uses: tirth8205/code-review-graph@v2.3.6
        with:
          github-token: ${{ secrets.GITHUB_TOKEN }}

This configuration posts a sticky comment on every PR containing the risk analysis table without blocking merges.

Adding Risk Gates

To prevent merging of high-risk changes, add the fail-on-risk input:

- uses: tirth8205/code-review-graph@v2.3.6
  with:
    github-token: ${{ secrets.GITHUB_TOKEN }}
    fail-on-risk: high

Valid values are none (default), high, or critical. When the overall risk score meets or exceeds the threshold, the workflow exits with a non-zero status, failing the check.

Handling Fork Pull Requests

The default GITHUB_TOKEN in fork-based PRs has read-only permissions for security reasons. To post comments safely from forks, implement a two-workflow pattern:

Analysis workflow (runs on the fork):

- uses: tirth8205/code-review-graph@v2.3.6
  with:
    github-token: ${{ secrets.GITHUB_TOKEN }}
    comment: false
    fail-on-risk: none

Trusted workflow (runs on base repository via workflow_run):

on:
  workflow_run:
    workflows: ["code-review-graph"]
    types:
      - completed
jobs:
  post-comment:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/download-artifact@v4
        with:
          name: crg-comment
      - uses: actions/github-script@v7
        with:
          script: |
            const fs = require('fs');
            const comment = fs.readFileSync('crg-comment.md', 'utf8');
            await github.rest.issues.createComment({
              owner: context.repo.owner,
              repo: context.repo.repo,
              issue_number: context.payload.workflow_run.pull_requests[0].number,
              body: comment
            });

The analysis workflow uploads the crg-comment.md artifact, which the trusted workflow downloads and posts using elevated permissions.

Configuration Examples

Analysis Without Comments

For custom reporting pipelines where you only need the JSON output:

- uses: tirth8205/code-review-graph@v2.3.6
  with:
    github-token: ${{ secrets.GITHUB_TOKEN }}
    comment: false
    fail-on-risk: none

This skips the comment posting step while still caching the graph database for subsequent runs.

Critical Risk Gate for Production Branches

Protect your main branch by failing on any critical risk changes:

name: risk-gate
on:
  pull_request:
    branches: [main]
permissions:
  contents: read
  pull-requests: write
jobs:
  review:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v7
      - uses: tirth8205/code-review-graph@v2.3.6
        with:
          github-token: ${{ secrets.GITHUB_TOKEN }}
          fail-on-risk: critical
          comment: true

Core Components and Source Files

The implementation relies on three primary components working in sequence:

• action.yml – Defines the composite action interface, inputs (github-token, fail-on-risk, comment), caching strategy, and step orchestration.
• scripts/render_pr_comment.py – Handles the conversion of JSON analysis results to markdown, implements the hidden marker logic for comment updating, and enforces the exit code 3 behavior for risk threshold violations.
• docs/GITHUB_ACTION.md – Contains comprehensive documentation, risk level mappings, and the fork-safe workflow patterns.

The action uses gh api commands to either PATCH existing comments (identified by the hidden HTML marker) or POST new ones, ensuring only one persistent comment exists per pull request regardless of how many times the workflow runs.

Summary

• Risk-scored PR reviews analyze changed symbols using a dependency graph and assign scores from 0 (low risk) to 1 (critical).
• The composite action in action.yml handles setup, caching, analysis, and reporting in a single reusable unit.
• Configure fail-on-risk with values high or critical to block merges based on calculated risk thresholds.
• Use comment: false for analysis-only runs or when implementing custom reporting workflows.
• Fork PRs require a two-workflow pattern separating analysis (untrusted) from comment posting (trusted) to handle permission limitations safely.
• The scripts/render_pr_comment.py script exits with status 3 when risk gates are breached, providing clear CI failure signaling.

Frequently Asked Questions

How does the action determine which code changes to analyze?

The action detects the base reference using github.base_ref for pull request events, or falls back to HEAD~1 for other trigger types. It then runs code-review-graph detect-changes --base "$CRG_BASE" to identify symbols modified between the base and HEAD commits, calculating risk scores only for the changed functions, classes, and modules.

What happens when the risk threshold is exceeded?

When fail-on-risk is set to high or critical and the overall risk score meets that level, the render_pr_comment.py script exits with status code 3. This non-zero exit code causes the GitHub Actions job to fail, which prevents the pull request from being merged if the workflow is configured as a required status check.

Can I use this action with private repositories?

Yes, the action works with private repositories using the standard GITHUB_TOKEN secret. The token requires contents: read permission to checkout code and pull-requests: write permission to post or update the sticky comment. No third-party services or additional authentication tokens are required.

How does the caching mechanism work?

The action uses actions/cache@v6 to store the .code-review-graph/ directory between runs. The cache key includes the schema version (schema9), ensuring that incompatible graph formats are automatically rebuilt rather than restored. On cache hits, the action performs incremental updates using code-review-graph update; on misses, it executes a full build operation.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}