How the Cross-Validation Tool Identifies Discrepancies in Financial Data Sources

The cross-validation tool converts all input values to exact Decimal objects, computes the median as a reference point, and flags any source whose absolute percentage deviation exceeds a configurable tolerance (default 2%) with a ❌ marker.

The xbtlin/ai-berkshire repository provides a specialized cross-validation tool designed to detect data inconsistencies before they corrupt financial models. When aggregating metrics like revenue or EBITDA from multiple providers—such as annual reports, Yahoo Finance, or alternative data feeds—analysts face the risk of propagating erroneous figures into valuation calculations. The tool implements a rigorous statistical workflow in tools/financial_rigor.py that normalizes inputs, establishes a consensus reference, and quantifies deviations to surface actionable discrepancies.

The Cross-Validation Algorithm in tools/financial_rigor.py

The core validation logic resides between lines 73 and 104 of tools/financial_rigor.py. When invoked via the cross_validate function, the tool executes a six-step pipeline that eliminates floating-point noise and applies statistical outlier detection.

Input Normalisation with Exact Decimal Conversion

The process begins by sanitizing raw inputs to prevent rounding artifacts from masking true discrepancies. The source_values dictionary is passed through the exact() helper (lines 73-75), which transforms every numeric entry into a precise Decimal object. This step is critical because standard floating-point arithmetic can introduce micro-errors that trigger false positives during comparison.

Reference Value Computation via Median

Rather than using a simple arithmetic mean, the tool sorts all normalized values and selects the median as the reference benchmark (lines 78-81). This approach reduces the influence of extreme outliers that might skew the consensus, ensuring that the reference point represents the statistical center of the observed data.

Deviation Calculation and Threshold Enforcement

For each source, the tool calculates the absolute percentage deviation from the median using the formula:

dev = abs(float(val) - median) / median * 100

This deviation is immediately compared against the tolerance_pct parameter, which defaults to 2% (lines 86-92). Sources exceeding this threshold receive a ❌ flag in the output report, while those within tolerance display ✅. The binary consistency indicator helps analysts visually identify problematic data points at a glance.

Consensus Reporting and Return Structure

The final output generation (lines 94-104) produces both human-readable terminal output and a structured dictionary. The report includes:

• The total number of sources evaluated
• The median reference value (labeled as consensus)
• Individual source values with their calculated deviations
• A consistency verdict flagging whether all sources align within tolerance

The function returns a dictionary containing the consensus median value and an all_consistent Boolean indicating perfect alignment (lines 100-104). When inconsistencies are detected, the tool explicitly advises prioritizing "company annual report / exchange data" as the authoritative source (line 99).

Practical Implementation Examples

Command-Line Interface Usage

Analysts can invoke the validator directly from the terminal using the cross-validate subcommand. The following example compares revenue figures (in 亿 units) across three distinct providers:

# Basic usage – compare three sources for revenue (unit: 亿)

python3 tools/financial_rigor.py cross-validate \
    --field revenue \
    --values '{"年报": 7518, "Yahoo": 7500, "StockAnalysis": 7520}' \
    --unit 亿

This execution normalizes the inputs, calculates the median (7518), identifies that Yahoo's value (7500) deviates by approximately 0.24%, and returns a ✅ verdict if the tolerance is set to 2%.

Programmatic Integration

For automated pipelines, import the cross_validate function directly into Python scripts:

# Programmatic use inside another script

from tools.financial_rigor import cross_validate

values = {"AnnualReport": 7518, "Yahoo": 7500, "Morningstar": 7520}
result = cross_validate("Revenue", values, unit="亿", tolerance_pct=2.0)

print(result["consensus"])          # → 7518 (median)

print(result["all_consistent"])     # → False (Yahoo deviates >2%)

In this scenario, the function returns the consensus median of 7518 but sets all_consistent to False because the Yahoo figure falls outside the acceptable deviation window.

Summary

• Exact Decimal conversion eliminates floating-point rounding errors that could trigger false discrepancy alerts.
• Median-based referencing provides outlier-resistant consensus calculation compared to arithmetic means.
• Configurable 2% tolerance allows adjustment based on data volatility while providing clear ✅/❌ visual indicators.
• Dual output format supplies both terminal reports for human review and JSON-serializable dictionaries for API integration.
• Source hierarchy guidance explicitly recommends annual report and exchange data as the gold standard when conflicts arise.

Frequently Asked Questions

What tolerance threshold does the cross-validation tool use by default?

The tool applies a default tolerance of 2% when comparing source deviations against the median reference. This value is configurable via the tolerance_pct parameter in both the CLI and Python API, allowing analysts to tighten or relax the validation criteria based on the specific volatility of the financial metric being examined.

Why does the tool use median instead of mean for the reference value?

The implementation selects the median rather than the mean to minimize outlier sensitivity. Financial data sources occasionally contain anomalous values due to reporting delays or unit conversion errors; the median provides a robust central tendency measure that remains stable even when individual sources report extreme values, ensuring the reference point truly represents the consensus.

How does the tool handle floating-point precision issues?

Before any deviation calculations occur, the exact() helper function converts all input values to Decimal objects (lines 73-75). This conversion eliminates binary floating-point representation errors that could accumulate during percentage deviation calculations, ensuring that discrepancies smaller than 0.01% are detected accurately without artificial noise.

What should analysts do when the tool flags inconsistencies?

When the cross_validate function detects deviations exceeding the tolerance threshold, it outputs explicit guidance at line 99 recommending that analysts prioritize "company annual report / exchange data" as the most reliable source. This hierarchy leverages the regulatory audit requirements of official filings over third-party aggregators, ensuring downstream valuation models use the most authoritative available figures.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}