How to Debug Review Quality Issues and Tool Call Traces in Open Code Review

Enable OCR_DEBUG=1 and use ocr rules check to trace which rules apply to specific files, then inspect session.jsonl and panic stack traces in internal/llmloop/pool.go and internal/agent/agent.go to diagnose missing or low-quality review comments.

The Alibaba Open Code Review (OCR) toolchain provides multiple built-in debugging mechanisms for investigating why review comments are missing, malformed, or unexpected. Whether you're tracing rule resolution, inspecting LLM tool-call outputs, or capturing panic stack traces, the codebase exposes concrete entry points for root-cause analysis.

---

Verify Rule Matching with ocr rules check

The fastest way to confirm that OCR applies the rule you expect is the rules check subcommand.

Basic Usage

ocr rules check src/main/java/com/example/Foo.java

This outputs the rule source (system, global, project, or custom), the matched glob pattern, and the full rule JSON. You can also override with a custom rule file:

ocr rules check --rule custom.json src/main/resources/mapper/UserMapper.xml

Implementation Details

The command is defined in [cmd/opencodereview/rules_cmd.go](https://github.com/alibaba/open-code-review/blob/main/cmd/opencodereview/rules_cmd.go):

• Lines 25–34 define the CLI flags and argument parsing
• Lines 58–73 invoke the DetailResolver via ResolveDetail and print the resolved configuration

The resolver itself lives in the internal/config/rules package. It walks rule layers in priority order (system → global → project → custom) and returns the first match. If no match reports, verify that:

• File paths are normalized using strings.ToLower
• Glob patterns in your rule JSON match the actual file path

---

Capture Tool Call Panic Traces

When an LLM tool call crashes during review, OCR captures the full stack trace to stderr. Two primary locations emit these diagnostics:

Worker Pool Panics in internal/llmloop/pool.go

The comment post-processing worker pool (lines 96–99) recovers from panics and prints:

defer func() {
    if r := recover(); r != nil {
        fmt.Fprintf(stdout.Writer(),
            "[ocr] CommentWorkerPool panic: %v\n%s\n", r, debug.Stack())
    }
}()

Subtask Panics in internal/agent/agent.go

The per-file review runner (lines 597–599) handles panics in individual file subtasks:

defer func() {
    if r := recover(); r != nil {
        fmt.Fprintf(stdout.Writer(),
            "[ocr] Subtask panic for %s: %v\n%s\n", d.NewPath, r, debug.Stack())
    }
}()

Expected Output Format

[ocr] CommentWorkerPool panic: runtime error: index out of range [5] with length 3
goroutine 42 [running]:
runtime/debug.Stack()
        /usr/local/go/src/runtime/debug/stack.go:24 +0x5e
...

Set OCR_DEBUG=1 to ensure these messages surface in your terminal. The environment variable is checked in internal/stdout/stdout.go before printing, so it has no effect on normal operation.

---

Inspect Raw LLM Outputs via Session JSONL

After each review run, OCR persists the raw LLM responses to a session.jsonl file. The comment in [internal/scan/agent.go](https://github.com/alibaba/open-code-review/blob/main/internal/scan/agent.go) (line 1020) explicitly references this for "debug bad outputs from session JSONL".

How to Read the Session File

cat ocr_session_2024-04-01.jsonl | jq .

Key fields to examine:

• toolCalls — the structured tool invocations the model attempted
• comment — generated review content before post-processing
• rawResponse — complete model output including any malformed JSON

This lets you verify whether the model produced valid tool-call payloads or hallucinated incorrect arguments.

---

Diagnose Rule Filtering and File Exclusion

Files silently excluded by rule filters will never trigger tool calls. The FileFilter stored in Args.FileFilter ([internal/agent/agent.go](https://github.com/alibaba/open-code-review/blob/main/internal/agent/agent.go), lines 70–73) controls include/exclude patterns from rule JSON.

To inspect the effective filter:

1. Add temporary fmt.Printf logging in runRulesCheck
2. Or run ocr rules check on a known-included file and compare with an excluded one

---

Debug Workflow Summary

Symptom	Diagnostic Action	Key File
Rule not applying as expected	Run ocr rules check <file>	cmd/opencodereview/rules_cmd.go
Crash during review with no context	Check stderr for [ocr] ... panic: messages	internal/llmloop/pool.go, internal/agent/agent.go
Missing or strange comments	Inspect session.jsonl with jq	internal/scan/agent.go
File silently skipped	Verify FileFilter patterns	internal/agent/agent.go

---

Summary

• Use ocr rules check to trace rule resolution through system, global, project, and custom layers
• Enable OCR_DEBUG=1 to surface panic stack traces from worker pools and subtask runners
• Read session.jsonl with jq to audit raw LLM tool-call payloads
• Check FileFilter configuration when files are unexpectedly excluded from review

These debugging mechanisms in alibaba/open-code-review give you complete visibility into the review pipeline, from rule matching through final comment generation.

---

Frequently Asked Questions

Why does ocr rules check show no matching rule for my file?

The resolver normalizes paths with strings.ToLower and matches against glob patterns in priority order. Verify your rule's glob pattern covers the file path, and check that no higher-priority rule is matching first. Use --rule to test with a specific custom rule file.

How do I see the exact JSON the LLM returned?

Open the session.jsonl file generated after your review run and pipe it through jq. Look for the toolCalls or rawResponse fields to see unprocessed model output, including any malformed tool invocations.

What does a [ocr] Subtask panic error indicate?

This means the per-file review goroutine crashed—typically due to an unexpected nil pointer, index out of bounds, or malformed tool response. The stack trace printed to stderr originates from internal/agent/agent.go lines 597–599 and shows exactly which function chain failed.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}