Setting up GitHub Action for Automated Risk-Scored PR Reviews with code-review-graph

code-review-graph provides a composite GitHub Action that builds a local Tree-sitter knowledge graph, computes a 0-1 risk score for pull request changes, and posts a sticky markdown comment without ever transmitting source code or tokens outside the runner.

This setup targets engineering teams who want AI-assisted PR reviews while maintaining strict data privacy. The action runs entirely on your CI infrastructure, analyzing code structure through a SQLite-based graph database that persists between builds. Unlike cloud-based review tools, this approach keeps diffs, source files, and repository metadata local to the GitHub Actions runner.

Architecture of the Composite Action

The workflow is defined in action.yml at the repository root. It orchestrates seven discrete phases: dependency installation, cache restoration, graph building or updating, risk analysis, markdown rendering, sticky comment upsertion, and optional risk-gate enforcement.

Local-First Graph Construction

The action invokes code-review-graph build to parse your repository using Tree-sitter. This generates .code-review-graph/graph.db, a SQLite database containing nodes for functions, classes, imports, and call edges. For incremental updates, the action runs code-review-graph update, which re-parses only files differing from the PR base reference stored in the CRG_BASE environment variable.

Risk Analysis Engine

The core scoring happens in code_review_graph/changes.py:compute_risk_score. The detect-changes command walks the graph to produce a JSON payload containing:

• Changed symbols with individual risk contributions
• Affected execution flows mapped through the graph
• Test coverage gaps identified in modified paths
• Context savings metrics estimating token reduction for AI assistants

The final output is a normalized overall risk score between 0 and 1.

Sticky Comment Workflow

The Python script scripts/render_pr_comment.py converts the JSON analysis into a GitHub-flavored markdown report. It injects a hidden HTML marker (<!-- code-review-graph-report -->) into the comment body. On subsequent pushes, the action queries existing comments via the gh CLI, identifies the marker, and PATCHes the existing comment rather than creating noise with new posts.

Configuring the GitHub Action Workflow

Minimal Setup

Create .github/workflows/code-review-graph.yml to run the action on every pull request. This configuration requires only read access to contents and write access to pull requests.

name: code-review-graph

on:
  pull_request:

permissions:
  contents: read
  pull-requests: write

jobs:
  review:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: tirth8205/code-review-graph@v2.3.6
        with:
          github-token: ${{ secrets.GITHUB_TOKEN }}

The repository dog-foods this exact pattern in .github/workflows/pr-review.yml.

Enforcing Risk Gates

To block merging when risk exceeds tolerance levels, set the fail-on-risk input. The action exits with code 3 when thresholds are breached: high (≥ 0.70) or critical (≥ 0.85).

      - uses: tirth8205/code-review-graph@v2.3.6
        with:
          github-token: ${{ secrets.GITHUB_TOKEN }}
          fail-on-risk: high

This transforms the review job into a required merge gate, failing the workflow when the overall risk score from compute_risk_score indicates dangerous change clusters.

Disabling Comments for Fork Safety

For workflows triggered by forked repositories where token permissions are restricted, disable comment posting while retaining the JSON artifact.

      - uses: tirth8205/code-review-graph@v2.3.6
        with:
          github-token: ${{ secrets.GITHUB_TOKEN }}
          comment: false

You can inspect the markdown locally using the same rendering logic:

code-review-graph detect-changes --base origin/main | \
python scripts/render_pr_comment.py --output report.md

Caching Strategy for Incremental Builds

The action leverages actions/cache to persist the .code-review-graph/ directory between runs. The cache key embeds the graph schema version (schema9) combined with a hash of common lockfiles. This guarantees that schema migrations invalidate the cache, forcing a fresh rebuild while preserving performance for routine PRs.

As documented in docs/GITHUB_ACTION.md, this strategy ensures that code-review-graph update operates only on changed files, reducing execution time from minutes to seconds on large repositories.

Downstream Workflow Patterns

For repositories requiring strict permission separation, split the workflow into an untrusted analysis job and a trusted publishing job. The analysis job uploads the rendered comment as an artifact; the publishing job downloads it and posts via the GitHub API.

name: Publish PR Review Comment
on:
  workflow_run:
    workflows: [code-review-graph]
    types: [completed]

permissions:
  contents: read
  pull-requests: write

jobs:
  publish:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/download-artifact@v4
        with:
          name: crg-report
      - run: |
          COMMENT=$(cat crg-comment.md)
          PR=$(cat pr-number.txt)
          gh api \
            "repos/${GITHUB_REPOSITORY}/issues/${PR}/comments" \
            -f body="${COMMENT}"

This pattern appears in .github/workflows/pr-review-comment.yml and prevents privilege escalation from fork PRs.

Summary

• code-review-graph operates as a composite action defined in action.yml, requiring no external API calls or SaaS dependencies.
• Risk scoring occurs locally via code_review_graph/changes.py:compute_risk_score, producing a 0-1 metric based on graph topology and change impact.
• The sticky comment mechanism uses an HTML marker to update existing PR comments rather than creating spam.
• Risk gates enforce quality bars by exiting with code 3 when scores exceed 0.70 (high) or 0.85 (critical).
• Schema-versioned caching keeps incremental builds fast while ensuring database compatibility across updates.

Frequently Asked Questions

Where does the risk score calculation happen?

The calculation is implemented in code_review_graph/changes.py:compute_risk_score. This function analyzes the delta between the PR base and head within the Tree-sitter graph, weighting factors like cyclomatic complexity changes, call graph depth, and test coverage proximity to generate the normalized 0-1 score.

Can I run this on self-hosted runners?

Yes. The action runs entirely on the CI runner with no outbound data transmission. Self-hosted runners are fully supported provided they have Python 3.10+, Node.js for Tree-sitter bindings, and the gh CLI available in the PATH.

What happens if the graph schema changes?

The cache key includes the schema version (currently schema9). When the repository updates to a newer version of code-review-graph that modifies the database schema, the cache miss triggers a fresh code-review-graph build, preventing corruption of the SQLite graph at .code-review-graph/graph.db.

How does the action handle force-pushes?

The detect-changes command accepts a --base parameter referencing CRG_BASE. When the action detects a force-push or rebase, it invalidates the incremental update and rebuilds the graph subgraph for the modified files, ensuring the risk analysis always reflects the current PR state rather than stale cached data.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}