How to Cross-Validate Financial Data Across Multiple Sources in AI Berkshire

The AI Berkshire framework validates financial metrics by fetching data from two independent sources, computing a median consensus, and flagging deviations exceeding a 2% tolerance to ensure investment-grade accuracy.

Cross-validation of financial data across multiple sources is essential for building auditable investment-research workflows. The AI Berkshire repository implements a declarative validation pipeline that automatically compares key metrics like revenue, net profit, and EPS from distinct providers before they reach downstream valuation models. This architecture ensures that every figure used in /investment-research or /earnings-review commands has survived a quantitative second-opinion check.

Architecture of the Cross-Validation Pipeline

The validation system operates through four distinct layers that separate policy definition from execution.

Skill Layer

The Skill Layer declares what must be validated. In skills/financial-data.md, the framework defines a data-source hierarchy for each market (US, HK, CN) and specifies that every key metric requires verification from two independent providers. This layer also configures error-rate thresholds and formatting rules for consensus badges (✅, ⚠️, ❌).

Tool Layer

The Tool Layer executes the how. The cross_validate function in tools/financial_rigor.py performs the statistical comparison. It converts incoming values to Decimal objects for exact arithmetic, computes a median reference point, and flags any source whose absolute deviation exceeds the configurable tolerance.

Agent Layer

Parallel agents invoke the toolkit automatically when a skill reaches a validation checkpoint. When processing a company profile, the agent fetches numbers from tools/stock_screener.py (Macrotrends, Yahoo) and tools/xueqiu_scraper.py (for Chinese equities), then triggers cross_validate before proceeding to valuation calculations.

Data Layer

Raw JSON values harvested by web scrapers reside in data/fundamentals.json. This layer stores the unvalidated inputs that the toolkit compares before producing a consensus value for downstream consumption.

The cross_validate Function Implementation

The core validation logic resides in tools/financial_rigor.py. The function accepts a dictionary of source values, computes a median consensus, and returns a consistency flag.

def cross_validate(field_name, source_values: dict, unit="", tolerance_pct=2.0):
    # Convert every value to a Decimal for exact arithmetic

    values = {k: exact(v) for k, v in source_values.items()}
    # Median serves as the unbiased consensus reference

    median = compute_median(values)
    # Flag any source whose absolute deviation from the median exceeds tolerance_pct

    for src, val in values.items():
        dev = abs(float(val) - median) / median * 100 if median else 0
        status = "✅" if dev <= tolerance_pct else "❌"
    # Return consensus and overall consistency flag

    return {"consensus": median, "all_consistent": all(dev <= tolerance_pct for dev in deviations)}

Key parameters:

• field_name: The financial metric being validated (e.g., "Revenue", "EPS").
• source_values: Dictionary mapping provider names to raw numeric values.
• tolerance_pct: Default threshold of 2.0% for acceptable deviation from the median.
• unit: Optional string (e.g., "亿") for display formatting.

Practical Usage Examples

Command-Line Validation

You can invoke the validator directly from the terminal to audit figures before importing them into research workflows.

python3 tools/financial_rigor.py cross-validate \
  --field revenue \
  --values '{"年报": 7518, "Yahoo": 7500, "StockAnalysis": 7520}' \
  --unit "亿" \
  --tolerance 2.0

Expected output:

============================================================
交叉验证: revenue (Cross-Validation)
============================================================

  数据来源数: 3
  参考中位数: 7518.00 亿

  ✅ Macrotrends        : 7518.00 亿  (偏差 0.00%)
  ✅ StockAnalysis      : 7520.00 亿  (偏差 0.03%)
  ✅ Yahoo              : 7500.00 亿  (偏差 0.24%)

  ✅ 所有来源偏差 ≤ 2.0%, 数据一致

  共识值 (加权中位数): 7518.00 亿

Embedding in Investment Research Skills

Skills import the toolkit to enforce validation before using any financial metric in valuation models.

from tools.financial_rigor import cross_validate

# Inside a skill implementation (e.g., investment-research)

source_vals = {
    "Macrotrends": fetch_macrotrends(ticker, "revenue"),
    "Yahoo": fetch_yahoo(ticker, "revenue")
}
result = cross_validate("Revenue", source_vals, unit="亿")

if not result["all_consistent"]:
    raise ValueError("Revenue data discrepancy exceeds tolerance")

consensus_revenue = result["consensus"]

# Use consensus_revenue for DCF or comparable valuation models

Report Integration

When the /investment-research skill generates output, it displays the consensus value alongside provenance badges. The formatting rules in skills/financial-data.md mandate this transparency:

收入：7,518亿元 ✅
  - Macrotrends: 7,518亿元
  - Yahoo: 7,500亿元
  - 误差: 0.24%

This automated annotation ensures that analysts can trace every number back to its dual-source verification.

Configuration and Tolerance Settings

The data-source hierarchy and validation rules are declarative. The skills/financial-data.md file specifies which providers serve as primary and secondary sources for each market, ensuring that Chinese A-share metrics always cross-reference against xueqiu_scraper.py data while US equities prioritize stock_screener.py outputs. When deviations exceed the tolerance defined in the skill specification, the pipeline raises a warning and requires manual review of the original filings rather than propagating suspect data.

Summary

• Cross-validation is mandatory for all key financial metrics in AI Berkshire, enforced at the skill level via skills/financial-data.md rules.
• The cross_validate function in tools/financial_rigor.py computes a median consensus and flags sources deviating more than 2% from that reference.
• Execution is automatic through parallel agents that fetch data from tools/stock_screener.py and tools/xueqiu_scraper.py, storing raw inputs in data/fundamentals.json.
• Consensus values are injected into downstream skills like /investment-research only after passing the dual-source sanity check.
• Failures block the pipeline, requiring manual review before any valuation calculations proceed.

Frequently Asked Questions

What tolerance threshold does AI Berkshire use for financial data validation?

AI Berkshire defaults to a 2% tolerance for cross-validation of financial data across multiple sources. This threshold is configurable via the tolerance_pct parameter in cross_validate, but the skill specification in skills/financial-data.md enforces the 2% standard for all automated investment-research workflows to maintain consistency.

How does the cross_validate function handle currency precision?

The function uses exact Decimal arithmetic rather than floating-point math. It converts all input values via an exact() helper before computing the median, preventing rounding errors that could skew deviation calculations when dealing with large financial figures (e.g., billions in CNY).

Where are the source data hierarchies defined in AI Berkshire?

Market-specific source priorities and error thresholds are defined in skills/financial-data.md. This file specifies that US equities must validate against independent providers like Macrotrends and Yahoo, while Chinese equities require cross-checking against Xueqiu data, ensuring geographic coverage matches local reporting standards.

What happens when financial data exceeds the validation tolerance?

When any source deviates beyond the configured tolerance from the median consensus, the pipeline raises a validation warning and blocks automatic progression to valuation models. The system requires manual review of the original filings (e.g., annual reports) before the metric can be marked as verified and injected into downstream outputs.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}