How the code-review-graph GitHub Action Computes Risk Scores and Posts PR Review Comments

The GitHub Action runs a graph‑based analyser that weights call‑graph impact, test coverage, and security signals to produce a 0‑to‑1 risk score for each changed function, then publishes a Markdown summary to the PR via the GitHub API.

The tirth8205/code-review-graph repository automates code review by evaluating pull requests for architectural risk before they reach production. When a developer opens or updates a PR, the Action invokes the code‑review‑graph CLI to map the codebase, isolate affected nodes, and compute quantitative risk assessments that are posted back as review comments. Below is the complete technical flow, from change detection to API submission.

Detecting Changes with the Review Tool

The workflow begins by calling the CLI entry point implemented in code_review_graph/tools/review.py. The command code-review-graph review constructs a comprehensive graph of the entire repository, then filters the graph to only those nodes that were added, modified, or deleted in the PR.

# Local execution mirrors the Action's internal call

code-review-graph review \
    --repo . \
    --pr 42 \
    --output report.json

The tool outputs a JSON report containing the isolated subgraph and metadata required for scoring.

Risk Score Calculation

The compute_risk_score Function

Actual risk computation lives in code_review_graph/tools/analysis_tools.py. The function compute_risk_score accepts a node and its context, returning a floating‑point value between 0 (no risk) and 1 (critical).

Risk Factors Weighed by the Algorithm

• Call‑graph impact – The number of incoming edges (callers) to the node; higher fan‑in elevates risk because regressions affect more of the system.
• Test coverage – A binary or scalar flag indicating whether the node is exercised by existing tests; uncovered nodes receive a penalty multiplier.
• Security relevance – Pattern matching against security‑related keywords in the function name, class name, or docstring (e.g., “auth”, “crypt”, “password”).
• Temporal churn (optional) – Commit frequency data that adjusts the score for files with high recent modification rates.

Aggregating Individual Scores into a PR‑Level Rating

After each affected node receives a score, the analyser aggregates them using either average or max‑pooling to produce an overall PR risk value. The final report categorises the PR into one of four levels—low, medium, high, or critical—and lists the top‑risk functions alongside any test‑coverage gaps.

Rendering and Posting the Review Comment

Markdown Generation in render_pr_comment.py

The helper script scripts/render_pr_comment.py ingests the JSON report and formats a rich Markdown comment. The output includes a risk‑level banner, a table of high‑risk functions, and actionable hints for mitigation.

# Inside scripts/render_pr_comment.py

import os
import json
from github import Github

with open("report.json") as f:
    report = json.load(f)

comment_body = render.render_pr_comment(report)   # Builds Markdown table

gh = Github(os.getenv("GITHUB_TOKEN"))
pr = gh.get_repo(repo).get_pull(pr_number)
pr.create_issue_comment(comment_body)

GitHub API Submission and Deduplication

The script submits the comment using the GitHub REST API endpoint repos/{owner}/{repo}/issues/{pr_number}/comments. To avoid noise, the logic first checks the PR for an existing bot‑generated marker (a HTML comment or specific signature) and skips posting if one is found.

Workflow Orchestration and Fail‑Fast Behaviour

The entire pipeline is defined in .github/workflows/pr-review-comment.yml. The workflow checks out the code, executes the analyser, and conditionally posts the comment.

# .github/workflows/pr-review-comment.yml (simplified)

steps:
  - uses: actions/checkout@v3
  - name: Run code-review-graph
    run: |
      python -m code_review_graph.main review \
        --repo ${{ github.workspace }} \
        --pr ${{ github.event.pull_request.number }} \
        --output report.json
  - name: Post PR comment
    run: |
      python scripts/render_pr_comment.py \
        --report report.json \
        --pr ${{ github.event.pull_request.number }}
    env:
      GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

The Action accepts a fail-on-risk input (e.g., high or critical). When the computed overall risk exceeds the configured threshold, the process exits with a non‑zero status, causing the GitHub check suite to fail and blocking the merge.

Summary

• Change detection is handled by code_review_graph/tools/review.py, which isolates PR‑specific nodes from the global codebase graph.
• Risk scoring occurs in code_review_graph/tools/analysis_tools.py via compute_risk_score, blending call‑graph impact, test coverage, security keywords, and optional churn into a 0‑to‑1 value.
• Aggregation averages or max‑pools individual scores to assign an overall low/medium/high/critical grade.
• Comment rendering is performed by scripts/render_pr_comment.py, which produces Markdown and posts it through the GitHub API while preventing duplicates.
• Workflow configuration in .github/workflows/pr-review-comment.yml supports a fail-on-risk threshold to enforce quality gates.

Frequently Asked Questions

What source file contains the core risk calculation logic?

The algorithm is implemented in code_review_graph/tools/analysis_tools.py inside the compute_risk_score function. This module imports graph metrics and coverage data to return a normalised risk float for any given code node.

How does the Action avoid posting duplicate comments on every commit?

Before calling the GitHub API, scripts/render_pr_comment.py scans the existing PR comments for a unique bot signature or HTML marker. If the signature is detected, the script exits silently without submitting a new comment.

Can the workflow block merging when risk is too high?

Yes. The workflow accepts a fail-on-risk input parameter. When set to high or critical, the Action compares the aggregated PR risk level to the threshold and exits with a non‑zero code if the threshold is breached, failing the check and preventing merge.

Which CLI arguments does the review command support?

The code-review-graph review command accepts --repo to specify the repository path, --pr to provide the pull request number, and --output to define the JSON report destination. These are the same arguments used inside the GitHub Action runner as shown in code_review_graph/tools/review.py.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}