How AI Berkshire Uses Cross-Validation for Financial Data in Research Reports

AI Berkshire automatically verifies every numeric financial metric by cross-validating values from at least two independent sources using a median-based consensus algorithm that flags discrepancies exceeding a configurable tolerance.

In the AI Berkshire research pipeline, every financial figure that appears in a valuation report is treated as a verified fact rather than an assumed value. The repository implements a rigorous Cross-Source Data Validation system in tools/financial_rigor.py that automatically reconciles conflicting data points from multiple providers before they reach downstream models. This prevents large language models from propagating source-specific errors or hallucinating numeric values that could distort investment recommendations.

Why Cross-Validation is Mandatory for Financial Data

Financial statements, market-price feeds, and analyst estimates often diverge by a few percent due to timing differences, currency conversions, or varying accounting policies. AI Berkshire requires at least two independent sources for each key metric—such as revenue, net profit, or cash reserves—and enforces a configurable tolerance threshold (default 2%) to flag material discrepancies. This automatic guardrail ensures that downstream valuation ratios like PE, PB, and FCF Yield are calculated from reconciled, internally consistent inputs rather than raw, potentially conflicting data.

The Cross-Validation Architecture

The validation workflow is implemented in tools/financial_rigor.py and invoked as a mandatory step in the investment-research pipeline defined in skills/investment-research.md.

Data Collection and Source Pairing

Before validation runs, autonomous agents scrape raw numbers from prescribed source pairs documented in skills/financial-data.md. For U.S. equities, the system might pair Macrotrends with StockAnalysis, while Hong Kong stocks use AASTools paired with Macrotrends ADR data. These pairings ensure that no single provider’s error can contaminate the final report without detection.

The Validation Algorithm

At the core of the system is the cross_validate function in tools/financial_rigor.py. The algorithm executes the following steps:

1. Precision Handling – Converts every incoming value to an exact Decimal to prevent floating-point drift.
2. Consensus Calculation – Computes the median of the source set (or the average of the two middle values for even counts) to establish a reference point.
3. Deviation Analysis – Calculates the percentage deviation of each source from the median.
4. Tolerance Checking – Compares deviations against the tolerance_pct parameter (default 2%). Sources exceeding the threshold are marked with ❌; those within tolerance receive ✅.
5. Result Packaging – Returns a JSON-compatible payload containing the consensus value (the median) and an all_consistent boolean indicating whether all sources passed the tolerance check.

CLI Interface and Execution

Researchers invoke the tool via command line during the data preparation phase:

python3 tools/financial_rigor.py cross-validate \
  --field revenue \
  --values '{"年报": 7518, "Yahoo": 7500, "StockAnalysis": 7520}' \
  --unit 亿

The --tolerance flag allows per-metric adjustments. For example, cash-flow figures might require tighter controls (--tolerance 1.0), while market-price data might accept looser thresholds.

Integration with the Research Pipeline

The validation result is not merely advisory—it is a hard gate in the workflow.

Blocking Unclean Data

If any source exceeds the tolerance threshold, the workflow forces manual inspection before proceeding to valuation calculations like verify-valuation or three-scenario. This prevents the LLM from silently using disputed figures in DCF models or ratio analysis.

Audit Trail and Transparency

The tool outputs a readable table listing each source, raw value, unit, and deviation percentage. This table is embedded verbatim into the “关键数据交叉验证记录” (Key Data Cross-Validation Record) appendix of the final research report, providing readers with a complete audit trail of the data-quality check.

Implementation Reference

The validation system spans four key files in the repository:

• tools/financial_rigor.py – Contains the cross_validate function implementing the median-based consensus algorithm, Decimal precision handling, and CLI interface.
• skills/investment-research.md – Defines the mandatory workflow step that enforces cross-validation before downstream valuation.
• skills/financial-data.md – Governs which external APIs constitute independent source pairs for each market segment.
• README.md – Documents the “多源交叉验证” (Multi-Source Cross-Validation) capability at the feature level.

Summary

• AI Berkshire treats every financial metric as a verified fact requiring at least two independent sources.
• The cross_validate function in tools/financial_rigor.py computes a median consensus and flags deviations exceeding the configurable tolerance (default 2%).
• Validation acts as a hard gate: discrepancies block downstream valuation workflows until manually resolved.
• Results are embedded in report appendices, creating a transparent audit trail for every numeric claim.
• Source independence rules are maintained in skills/financial-data.md and enforced via skills/investment-research.md.

Frequently Asked Questions

What happens if two financial sources disagree by more than 2%?

If the percentage deviation from the median exceeds the configured tolerance, the source is marked with ❌ and the workflow forces manual inspection before proceeding. The all_consistent flag in the JSON output will be false, blocking downstream valuation steps like verify-valuation until the discrepancy is resolved.

How does the tool handle currency precision and rounding errors?

The cross_validate function converts all incoming values to Python’s Decimal type with exact arithmetic, eliminating floating-point rounding errors. This ensures that percentage deviations are calculated with financial-grade precision suitable for auditing.

Can the tolerance threshold be adjusted for different financial metrics?

Yes. While the default tolerance is 2%, researchers can pass the --tolerance <pct> flag on the CLI to specify stricter or looser thresholds. For example, cash reserve figures might use --tolerance 1.0 for tighter control, while market capitalization might accept --tolerance 5.0.

Where is the cross-validation result stored in the final report?

The JSON output containing the consensus value and consistency status is embedded into the “关键数据交叉验证记录” appendix section. This appendix includes a formatted table showing each source, its raw value, unit, and deviation percentage, providing a complete audit trail for readers.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}