# How AI Berkshire Performs Multi-Source Cross-Validation for Financial Data

> AI Berkshire ensures financial data accuracy with multi-source cross-validation. Discover its automated statistical analysis and human-verified audits for reliable insights.

- Repository: [Xbt Lin/ai-berkshire](https://github.com/xbtlin/ai-berkshire)
- Tags: how-to-guide
- Published: 2026-07-06

---

**AI Berkshire validates financial data by cross-checking every critical figure against independent providers using a two-tier system: automated statistical median analysis in [`tools/financial_rigor.py`](https://github.com/xbtlin/ai-berkshire/blob/main/tools/financial_rigor.py) and strict human-verified dual-source audits in [`tools/report_audit.py`](https://github.com/xbtlin/ai-berkshire/blob/main/tools/report_audit.py), ensuring no single data point relies on just one source.**

AI Berkshire is an open-source research framework that guarantees quantitative accuracy through rigorous multi-source cross-validation. Instead of trusting single data feeds, the system programmatically cross-references every financial metric against independent providers to eliminate transcription errors and source-specific biases. The implementation centers on two Python utilities that combine statistical consensus algorithms with human-in-the-loop verification.

## Core Validation Components

### Statistical Consensus via cross_validate

Located in [`tools/financial_rigor.py`](https://github.com/xbtlin/ai-berkshire/blob/main/tools/financial_rigor.py) (lines 63-71), the `cross_validate` function accepts a dictionary of values from different sources and computes a median or weighted median as the reference point. It flags any source whose deviation exceeds a configurable tolerance—defaulting to **2%**—and returns a consensus value alongside an "all-consistent" boolean flag. This utility prints a concise comparison table showing per-source deviations, making it easy to spot outliers at a glance.

### Human-in-the-Loop Auditing via render_verdict

The `render_verdict` function in [`tools/report_audit.py`](https://github.com/xbtlin/ai-berkshire/blob/main/tools/report_audit.py) (lines 43-61) handles the strict verification layer. After analysts manually populate `fetched_value` and `fetched_value2` fields from dual independent sources (e.g., macrotrends paired with stockanalysis), the function calculates percentage differences using the internal `_pct_diff` helper. It applies a rigid **1% tolerance** threshold (`_TOLERANCE = 0.01`) to generate PASS/FAIL verdicts, surfacing WARN status when sources diverge beyond acceptable limits but not both failing.

## The Cross-Validation Workflow

The multi-source validation process follows a five-stage pipeline:

1. **Data Extraction** – The `report_audit` tool parses Markdown research reports and extracts financial data from tables, key-value lines, and bolded figures.

2. **Random Sampling** – The system selects a representative **15% subset** of all extracted data points for verification, balancing thorough coverage with manageable workload.

3. **Dual-Source Lookup** – For each sampled datum, analysts retrieve values from two independent providers hard-coded in the validation logic:
   - U.S. equities: `macrotrends.net` + `stockanalysis.com`
   - Hong Kong listings: `aastocks.com` + `macrotrends ADR`
   - A-shares: `eastmoney.com` + `cninfo.com.cn`

4. **Cross-Validation Execution** – Validation occurs through two parallel tracks:
   - **Programmatic**: Direct invocation of `cross_validate` with JSON payloads computes median reference values and flags 2% deviations.
   - **Human-Verified**: The `verdict` sub-command evaluates manually populated dual-source data against the strict 1% tolerance, checking `_pct_diff` calculations.

5. **Outcome Classification** – Results are categorized as:
   - **PASS**: All sources within 1% tolerance, report cleared for publication
   - **FAIL**: Any source exceeds threshold, triggering mandatory corrections
   - **WARN**: Single source deviation while other complies, indicating potential currency conversion or accounting standard mismatches requiring manual review

## Command-Line Usage Examples

Execute direct statistical validation using the `cross_validate` command:

```bash
python3 tools/financial_rigor.py cross-validate \
    --field "每股收益 (EPS)" \
    --values '{"Macrotrends":23.5,"StockAnalysis":23.6,"Yahoo":23.4}' \
    --unit "" \
    --tolerance 2.0

```

Expected output:

```

交叉验证: 每股收益 (EPS) (Cross-Validation)
  数据来源数: 3
  参考中位数: 23.50
  ✅ Macrotrends       : 23.50   (偏差 0.00%)
  ✅ StockAnalysis     : 23.60   (偏差 0.43%)
  ✅ Yahoo             : 23.40   (偏差 0.43%)
  ✅ 所有来源偏差 ≤ 2.0%, 数据一致
  共识值 (加权中位数): 23.50

```

Run the complete audit workflow from extraction to verdict:

```bash

# Step A: Extract sample from report

python3 tools/report_audit.py extract \
    --report reports/腾讯/腾讯-research-20260408.md \
    --dry-run

# Step B: Manually populate fetched_value and fetched_value2 fields

# Step C: Evaluate audit

python3 tools/report_audit.py verdict \
    --results '[{"id":1,"label":"营业收入","reported_value":7518,"unit":"亿","fetched_value":7518,"fetched_source":"macrotrends","fetched_value2":7500,"fetched_source2":"stockanalysis"}]'

```

Sample verdict output:

```

✅ 通过 [ 1] 营业收入               报告:      7518.00 亿
              macrotrends: 7518.00 (偏差 0.00%)  |  stockanalysis: 7500.00 (偏差 0.24%)
...
【准出】所有抽检数据通过，报告可发布。

```

## Summary

- **AI Berkshire implements multi-source cross-validation** through [`tools/financial_rigor.py`](https://github.com/xbtlin/ai-berkshire/blob/main/tools/financial_rigor.py) and [`tools/report_audit.py`](https://github.com/xbtlin/ai-berkshire/blob/main/tools/report_audit.py), ensuring no financial metric relies on a single data provider.
- **Statistical validation** uses median-based consensus with a default 2% tolerance in `cross_validate`, while **human verification** applies stricter 1% thresholds in `render_verdict`.
- The workflow **randomly samples 15%** of report data for dual-source verification against provider pairs like macrotrends/stockanalysis or eastmoney/cninfo.
- **Clear outcome classifications** (PASS/FAIL/WARN) trigger automatic publication clearance or mandatory manual review when sources diverge.
- All validation logic is **open-source and reproducible**, located in specific line ranges ([`financial_rigor.py`](https://github.com/xbtlin/ai-berkshire/blob/main/financial_rigor.py) lines 63-71, [`report_audit.py`](https://github.com/xbtlin/ai-berkshire/blob/main/report_audit.py) lines 43-61) for transparency.

## Frequently Asked Questions

### What tolerance thresholds does AI Berkshire use for validation?

The system employs tiered tolerances: `cross_validate` defaults to **2%** for statistical outlier detection, while `render_verdict` enforces a strict **1%** limit (`_TOLERANCE = 0.01`) for final audit approval. This dual-threshold approach catches significant discrepancies early while maintaining rigorous standards for publication.

### How does the sampling process work?

The `report_audit` tool performs **random sampling of 15%** of all extracted financial data points from a research report. This percentage strikes a balance between statistical significance and operational efficiency, ensuring coverage across the entire document without requiring verification of every single figure.

### Which data providers are integrated into the validation system?

Source pairs are hard-coded based on asset class: U.S. equities use **macrotrends.net** and **stockanalysis.com** (or **Yahoo**); Hong Kong securities pair **aastocks.com** with **macrotrends ADR**; mainland Chinese A-shares validate against **eastmoney.com** and **cninfo.com.cn**. These combinations ensure geographic and methodological diversity.

### Can I use cross_validate without the full audit workflow?

Yes. The `cross_validate` function in [`tools/financial_rigor.py`](https://github.com/xbtlin/ai-berkshire/blob/main/tools/financial_rigor.py) operates independently via command-line invocation. You can pass any JSON payload of source values using the `--values` argument to receive immediate consensus calculations and deviation reports, making it suitable for ad-hoc data verification outside the formal `report_audit` pipeline.