How to Set Up the GitHub Action for Risk-Scored PR Reviews with code-review-graph

The code-review-graph repository provides a composite GitHub Action that builds a Tree-sitter knowledge graph locally on the runner, calculates a 0-1 risk score for pull request changes, and posts a sticky markdown comment with the analysis—all without transmitting source code or tokens to external services.

Setting up a GitHub Action for risk-scored PR reviews enables automated, privacy-preserving code analysis that runs entirely on your CI infrastructure. The code-review-graph tool constructs a local knowledge graph using Tree-sitter, computes risk scores based on changed symbols and execution flows via compute_risk_score in code_review_graph/changes.py, and delivers incremental reviews through sticky PR comments. This implementation keeps all source files, diffs, and tokens on the runner while providing AI assistants with structured context.

Architecture of the Composite Action

The workflow defined in action.yml operates as a composite GitHub Action that executes entirely on the CI runner. This local-first architecture ensures that no source code or sensitive data leaves your infrastructure.

The action performs the following steps:

• Installs the tool and restores a cached SQLite graph from .code-review-graph/graph.db
• Builds or incrementally updates the graph using code-review-graph build or code-review-graph update
• Runs risk analysis via code-review-graph detect-changes --base to compute a 0-1 overall risk score
• Renders a markdown report using scripts/render_pr_comment.py
• Upserts a sticky PR comment using the gh CLI, identified by the hidden HTML marker <!-- code-review-graph-report -->
• Optionally enforces a risk gate that exits with code 3 if thresholds are exceeded

Configuring the Workflow

Minimal Setup

Create .github/workflows/code-review-graph.yml with the following configuration. This workflow runs on pull requests with read access to contents and write access to pull requests.

name: code-review-graph

on:
  pull_request:

permissions:
  contents: read
  pull-requests: write

jobs:
  review:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v7
      - uses: tirth8205/code-review-graph@v2.3.6
        with:
          github-token: ${{ secrets.GITHUB_TOKEN }}

This configuration is the same workflow used to dog-food the repository in .github/workflows/pr-review.yml.

Enabling Risk Gates

To make the action function as a merge gate, configure the fail-on-risk input. The action supports high (≥ 0.70) and critical (≥ 0.85) thresholds.

      - uses: tirth8205/code-review-graph@v2.3.6
        with:
          github-token: ${{ secrets.GITHUB_TOKEN }}
          fail-on-risk: high

When the overall risk score exceeds the threshold, the script exits with code 3, failing the job and blocking the merge.

Running Without Comments

For fork-only workflows or when you need to generate reports without posting to the PR, disable commenting while preserving the JSON output.

      - uses: tirth8205/code-review-graph@v2.3.6
        with:
          github-token: ${{ secrets.GITHUB_TOKEN }}
          comment: false

You can render the markdown locally using the same rendering script:

code-review-graph detect-changes --base origin/main |
python scripts/render_pr_comment.py --output report.md

Split Workflow for Fork Safety

To handle pull requests from forks securely, use a split workflow pattern where the analysis runs in one workflow and a trusted workflow publishes the comment.

name: Publish PR Review Comment
on:
  workflow_run:
    workflows: [code-review-graph]
    types: [completed]

permissions:
  contents: read
  pull-requests: write

jobs:
  publish:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/download-artifact@v4
        with:
          name: crg-report
      - run: |
          COMMENT=$(cat crg-comment.md)
          PR=$(cat pr-number.txt)
          gh api \
            "repos/${GITHUB_REPOSITORY}/issues/${PR}/comments" \
            -f body="${COMMENT}"

This pattern is documented in docs/GITHUB_ACTION.md and implemented in .github/workflows/pr-review-comment.yml.

Performance Optimization and Caching

The action caches the .code-review-graph/ directory between runs using actions/cache. The cache key incorporates the schema version (schema9) and a hash of common lockfiles, ensuring that schema changes invalidate the cache and trigger a fresh build. Incremental updates via code-review-graph update re-parse only files that differ from the PR base (CRG_BASE), minimizing runtime on subsequent commits.

Key Implementation Files

• action.yml – Defines the composite GitHub Action steps including installation, caching, graph building, risk analysis, and comment upsert.
• docs/GITHUB_ACTION.md – Human-readable documentation covering inputs, outputs, caching strategies, and security considerations.
• scripts/render_pr_comment.py – Pure-Python renderer that converts detect-changes JSON output into the sticky markdown comment and enforces risk gate logic.
• code_review_graph/changes.py – Contains the compute_risk_score function that calculates overall risk from changed symbols, affected execution flows, and test coverage gaps.
• .github/workflows/pr-review.yml – Example workflow demonstrating the action running on its own pull requests.
• .github/workflows/pr-review-comment.yml – Trusted publishing workflow for handling artifacts from fork-based PRs.

Summary

• Local-first execution: The composite action defined in action.yml runs entirely on the CI runner, ensuring source code and tokens never leave your infrastructure.
• Incremental analysis: The tool caches the SQLite graph in .code-review-graph/ and uses code-review-graph update to process only changed files.
• Risk scoring: The compute_risk_score function in code_review_graph/changes.py calculates a 0-1 score based on changed symbols, execution flows, and coverage gaps.
• Sticky comments: The renderer in scripts/render_pr_comment.py uses the HTML marker <!-- code-review-graph-report --> to update existing comments rather than creating new ones.
• Merge gating: Set fail-on-risk to high (≥ 0.70) or critical (≥ 0.85) to exit with code 3 and block merges when risk thresholds are exceeded.

Frequently Asked Questions

What permissions does the GitHub Action require?

The action requires contents: read to check out the repository and pull-requests: write to post or update the sticky comment containing the risk analysis report.

How does the action handle incremental updates versus full rebuilds?

The action caches the graph database and uses code-review-graph update to re-parse only files that differ from the base branch (CRG_BASE). If the cache misses or the schema version changes (e.g., schema9), it runs code-review-graph build to create a fresh graph from the entire codebase.

Can I use this action safely with pull requests from forks?

Yes, by using the split-workflow pattern documented in docs/GITHUB_ACTION.md. The analysis runs in the context of the pull request workflow with the comment input set to false, uploads an artifact, and a trusted workflow (.github/workflows/pr-review-comment.yml) with pull-requests: write permission publishes the comment separately using the gh CLI.

What do the risk scores mean and how are they calculated?

The overall risk score ranges from 0 to 1. According to code_review_graph/changes.py:compute_risk_score, the system evaluates changed symbols with individual risk ratings, affected execution flows, and test coverage gaps. Scores ≥ 0.70 trigger the "high" threshold, while scores ≥ 0.85 trigger the "critical" threshold when using the fail-on-risk input, causing the action to exit with code 3.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}