# How to Use the report_audit.py Tool for Pre-Publication Verification

> Learn how to use the report_audit.py tool for pre-publication verification. This Python script ensures your research reports contain only verified financial figures against trusted sources.

- Repository: [Xbt Lin/ai-berkshire](https://github.com/xbtlin/ai-berkshire)
- Tags: how-to-guide
- Published: 2026-07-03

---

**The [`report_audit.py`](https://github.com/xbtlin/ai-berkshire/blob/main/report_audit.py) tool implements a three-step audit workflow—extract, verify, and verdict—to ensure research reports contain only financial figures verified against trusted external sources like Macrotrends and StockAnalysis.**

The [`report_audit.py`](https://github.com/xbtlin/ai-berkshire/blob/main/report_audit.py) utility in the `xbtlin/ai-berkshire` repository is a zero-dependency Python script located at [`tools/report_audit.py`](https://github.com/xbtlin/ai-berkshire/blob/main/tools/report_audit.py) designed to prevent publication of inaccurate financial data. Utilizing high-precision arithmetic from [`tools/financial_rigor.py`](https://github.com/xbtlin/ai-berkshire/blob/main/tools/financial_rigor.py), it parses Markdown research reports, randomly samples approximately 15% of detectable financial figures, and validates them against authoritative sources with a strict 1% tolerance threshold before granting a PASS or FAIL verdict.

## Step 1: Extract Financial Data Points

The first step generates a randomized sample of financial claims from your report. Run the extraction command:

```bash
python3 tools/report_audit.py extract \
  --report reports/your-report-name.md \
  --ratio 0.15 \
  --seed 42 \
  --dry-run

```

The `extract` command invokes `extract_data_points()` (lines 55-66) to locate numerical values in tables, "label: value" rows, and bold-number lines. It then calls `sample_points()` (lines 29-36) to randomly select a subset of data points, defaulting to 15% of the total with a minimum of 3 and maximum of 30 points to ensure statistical coverage without excessive manual labor.

Use `--dry-run` to preview the selected points without generating output. Omitting this flag produces a JSON template to stdout containing entries with placeholders for `fetched_value` and `fetched_source`. Each entry includes metadata such as `id`, `label`, `reported_value`, `unit`, `line_number`, and `raw_text` to facilitate precise verification.

## Step 2: Populate the Verification Template

After extraction, manually verify each sampled data point against trusted sources. The JSON template structure appears as follows:

```json
[
  {
    "id": 1,
    "label": "营业收入 · 2023年",
    "reported_value": 7518,
    "unit": "亿",
    "line_number": 123,
    "raw_text": "营业收入：7518亿元",
    "fetched_value": null,
    "fetched_source": "",
    "fetched_value2": null,
    "fetched_source2": ""
  }
]

```

Fill the following fields for each entry:

- **`fetched_value`**: The primary verification value from sources like Macrotrends or StockAnalysis
- **`fetched_source`**: Name of the primary data source
- **`fetched_value2`** (optional): Secondary verification value for cross-reference
- **`fetched_source2`** (optional): Name of the secondary source

Save the completed JSON to a file (e.g., [`verification_results.json`](https://github.com/xbtlin/ai-berkshire/blob/main/verification_results.json)) for the final step.

## Step 3: Render the Final Verdict

Submit the populated verification data to receive a PASS or FAIL judgment:

```bash
python3 tools/report_audit.py verdict \
  --results "$(cat verification_results.json)" \
  --report "My Research Report" \
  --output-json

```

The `verdict` command executes `render_verdict()` (lines 53-70), which calculates relative percentage differences using `_pct_diff()` (lines 46-51). It applies a **1% tolerance** (`_TOLERANCE = 0.01` at line 43) to classify each data point as:

- **PASS**: Fetched value matches reported value within 1%
- **FAIL**: Deviation exceeds 1%
- **WARN**: Primary source passes but secondary source diverges significantly

The command outputs a color-coded table summarizing each item's status. With `--output-json`, it emits a machine-readable verdict dictionary (line 111) containing `verdict`, `pass_count`, `fail_count`, and `warn_count`. The script exits with status `0` for PASS and `1` for FAIL (line 115), enabling integration with CI pipelines.

## Automating the Verification Workflow

Combine all three steps into a shell script for automated pre-publication checks:

```bash
#!/usr/bin/env bash
set -euo pipefail

REPORT="reports/tesla/tesla-research-2023.md"

# Extract data points

JSON_TEMPLATE=$(python3 tools/report_audit.py extract \
  --report "$REPORT" \
  --ratio 0.15 \
  --seed 123)

# Populate verification data (manual step simulated here)

FILLED=$(echo "$JSON_TEMPLATE" | jq 'map(
  .fetched_value = 1234,
  .fetched_source = "macrotrends",
  .fetched_value2 = 1230,
  .fetched_source2 = "stockanalysis"
)')

# Render verdict

python3 tools/report_audit.py verdict \
  --results "$FILLED" \
  --report "Tesla Research – 2023"

```

This automation ensures every numerical claim is cross-checked against external sources before publication, with the non-zero exit code blocking merges if verification fails.

## Summary

- **Three-step workflow**: Extract financial data with `extract`, manually verify against trusted sources, then render a `verdict`
- **Statistical sampling**: By default samples 15% of data points (minimum 3, maximum 30) using `sample_points()` to balance thoroughness with efficiency
- **Strict tolerance**: The `_TOLERANCE = 0.01` constant enforces a 1% maximum deviation between reported and fetched values
- **CI-ready**: Exit codes (`0` for PASS, `1` for FAIL) and `--output-json` support seamless integration into publication pipelines
- **Zero dependencies**: The tool requires only Python 3 standard library, leveraging [`tools/financial_rigor.py`](https://github.com/xbtlin/ai-berkshire/blob/main/tools/financial_rigor.py) for precision arithmetic

## Frequently Asked Questions

### What is the 1% tolerance rule in report_audit.py?

The tool defines `_TOLERANCE = 0.01` at line 43 of [`tools/report_audit.py`](https://github.com/xbtlin/ai-berkshire/blob/main/tools/report_audit.py), meaning fetched values must fall within 1% of the reported figure to receive a PASS classification. This accounts for minor rounding differences or timing discrepancies between data sources while flagging material errors.

### How many data points does the tool sample?

The `sample_points()` function (lines 29-36) selects approximately 15% of all detected financial figures by default, with a hard minimum of 3 points and maximum of 30. You can adjust the proportion using the `--ratio` parameter, though the min/max constraints remain to ensure statistical validity without excessive manual verification.

### Can report_audit.py integrate with CI/CD pipelines?

Yes. The script exits with status code `0` when all sampled points PASS and `1` when any point FAILs (line 115). Combined with the `--output-json` flag for machine-readable results, you can embed the three-step workflow in GitHub Actions, GitLab CI, or other automation platforms to block publication of unverified reports.

### What file formats does the audit tool support?

The `extract_data_points()` function (lines 55-66) specifically parses Markdown report files (`.md`) to identify numerical values in tables, labeled rows, and bold text. The tool outputs JSON templates and accepts JSON verification data, making it compatible with standard Unix pipelines and automation tools.