Reporting and Analytics in Open Code Review: CLI, Telemetry, and CI Integration

Open Code Review provides comprehensive reporting and analytics capabilities including session-level CLI inspection, OpenTelemetry observability for LLM metrics, and machine-readable JSON exports for CI/CD integration.

The alibaba/open-code-review (OCR) repository ships with built-in reporting and analytics features that let development teams observe how AI reviews are performed, measure LLM resource consumption, and extract structured review data for downstream dashboards. These capabilities are implemented across the CLI layer, a local session store, and an OpenTelemetry-compatible telemetry subsystem.

Session-Level Reporting and CLI Inspection

Every review run—whether a diff-review, full-scan, or delegated mode execution—is persisted as a session in the local .ocr directory. This session-based architecture enables retrospective analysis and auditing of all AI-generated feedback.

Listing and Querying Review Sessions

The CLI provides first-class commands for inspecting historical review data. According to the source documentation in README.md (lines 44-50), you can enumerate past sessions and extract their contents:

# List all recent review sessions with metadata

ocr session list

# Display human-readable comments for a specific session

ocr session comments abcdef12345

These commands interact with the session storage layer implemented in the cmd package, where session handlers manage persistence and retrieval logic.

Filtering and Exporting Structured Data

For integration with bug trackers and analytics pipelines, OCR supports severity-based filtering and JSON export. The --severity flag accepts comma-separated levels (e.g., critical,high), while the --json flag returns machine-readable output:

# Export only critical and high-severity issues as JSON

ocr session comments \
    --severity critical,high \
    --json \
    abcdef12345 > high-issues.json

The JSON output contains structured fields including file, line, severity, message, and author, enabling precise downstream processing.

Real-Time Review Summaries

After each review execution, OCR prints a concise summary to stdout containing the total issues found, a severity breakdown, and an optional "more issues" hint pointing to the full JSON report. This summary generation logic is implemented in the CI wrapper examples, specifically in examples/gitlab_ci/post_review.py at line 295, where the throttling and summary logic aggregates results before posting to merge request discussions.

The summary provides immediate visibility into review outcomes without requiring additional CLI commands.

OpenTelemetry Integration for Observability

OCR implements first-class OpenTelemetry support for observability into every review execution. When enabled via configuration, the telemetry subsystem emits OTLP (OpenTelemetry Protocol) spans and metrics via both gRPC and HTTP exporters.

Telemetry Configuration and Exporters

The telemetry setup is handled in internal/telemetry/exporter.go (lines 95-180), where the InitExporters function configures OTLP exporters for both traces and metrics. Configuration parameters—including service name and endpoints—are defined in internal/telemetry/config.go (line 21).

The system captures quantitative metrics essential for cost control and performance tuning:

• LLM request count and latency
• Tokens sent and received
• Number of review comments created
• Error rates and retry counts

Implementing Telemetry in Go

To instrument your own Go programs that invoke OCR, initialize the telemetry exporters as demonstrated in the internal implementation:

package main

import (
    "context"
    "go.opentelemetry.io/otel"
    "github.com/alibaba/open-code-review/internal/telemetry"
)

func main() {
    // Load configuration from OCR's config (e.g., .ocr/config.yaml)
    cfg := telemetry.LoadConfig()
    
    // Initialize OTLP exporters for traces and metrics
    if err := telemetry.InitExporters(cfg); err != nil {
        panic(err)
    }

    // Start a traced review operation
    _, span := otel.GetTracerProvider().Tracer("ocr").Start(context.Background(), "review-run")
    defer span.End()
    
    // Subsequent OCR calls automatically generate spans and metrics
}

This integration allows teams to correlate review activity with other services in their observability stack using tools like Prometheus, Grafana, and Jaeger.

CI/CD Pipeline Reporting

When OCR executes within CI environments (GitHub Actions, GitLab CI, GitFlic CI, or Gerrit CI), it writes machine-readable JSON artifacts containing the complete session data. This enables downstream dashboards to ingest review metrics automatically without parsing CLI output.

The CI wrappers located in examples/gitlab_ci/post_review.py handle this artifact generation. At line 160, the script collects session comments and writes them to the CI job's artifact store, while line 295 manages the summary aggregation logic. These wrappers ensure that review data persists beyond the ephemeral CI runner environment.

Summary

Open Code Review's reporting and analytics architecture provides:

• Session persistence via the local .ocr directory with CLI query capabilities (ocr session list, ocr session comments)
• Structured export supporting severity filtering and JSON output for integration with external systems
• Real-time summaries displaying issue counts and severity distributions immediately after review completion
• OpenTelemetry integration exposing LLM latency, token consumption, and error metrics through OTLP exporters in internal/telemetry/exporter.go
• CI/CD artifacts generated by wrapper scripts like examples/gitlab_ci/post_review.py for automated pipeline analytics

Frequently Asked Questions

How do I export review comments as JSON in Open Code Review?

Use the ocr session comments command with the --json flag. For example: ocr session comments --json <session-id>. You can combine this with --severity critical,high to filter for specific issue levels before exporting, producing structured data suitable for ingestion into bug trackers or data warehouses.

What metrics does Open Code Review expose via OpenTelemetry?

OCR exposes LLM request counts, request latency, tokens sent and received, the number of review comments generated, and error/retry statistics. These metrics are emitted through OTLP exporters configured in internal/telemetry/exporter.go and can be consumed by any OpenTelemetry-compatible backend.

Where are review sessions stored locally?

Review sessions are persisted in the .ocr directory within your project root or home directory, depending on the configuration. The CLI commands ocr session list and ocr session comments interact with this local storage to retrieve historical review data without requiring external database dependencies.

Can I filter comments by severity when exporting data?

Yes. The ocr session comments command accepts a --severity flag that accepts comma-separated values such as critical,high,medium,low. When combined with the --json flag, this allows you to export only high-priority issues for automated alerting or compliance reporting workflows.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}