How to Test the Effectiveness of the financial_rigor.py Module in ai-berkshire

To test the effectiveness of the financial_rigor.py module, implement deterministic unit tests for each validation function—such as verify_market_cap, cross_validate, and benford_check—asserting both successful validations and proper threshold warnings, then verify CLI integration and edge-case handling.

The financial_rigor.py module in the xbtlin/ai-berkshire repository provides a zero-dependency Python toolkit for validating financial data using exact decimal arithmetic. Testing its effectiveness ensures that market-cap verifications, valuation metrics, and Benford’s law checks return deterministic, accurate results under various conditions.

Unit Testing Strategy for financial_rigor.py

Import and Direct Function Testing

Import the module directly using from tools import financial_rigor as fr to bypass the CLI wrapper and test functions in isolation. Supply deterministic input data with known expected outputs—for example, verify that verify_market_cap returns True when calculated price × shares falls within 5% of the reported value, and False when deviation exceeds this threshold.

Assert on return values to validate both success and failure paths. For cross_validate, provide multiple data sources and verify that the function correctly identifies outliers when they exceed the specified tolerance percentage (default 2%). Capture stdout using io.StringIO to verify that user-facing messages such as "✅ 验证通过" or "❌ 警告" appear correctly.

Sample pytest Implementation

Create a test file in a tests/ directory to validate core functionality:

# tests/test_financial_rigor.py

import json
import io
import sys

from tools import financial_rigor as fr


def capture_stdout(func, *args, **kwargs):
    """Utility to capture printed output."""
    captured = io.StringIO()
    sys_stdout = sys.stdout
    sys.stdout = captured
    try:
        func(*args, **kwargs)
    finally:
        sys.stdout = sys_stdout
    return captured.getvalue()


def test_verify_market_cap_success():
    # price * shares = 100 * 1_000_000 = 100_000_000

    out = capture_stdout(fr.verify_market_cap, price=100, shares=1_000_000,
                         reported=100_050_000, currency="USD")
    assert "✅ 验证通过" in out
    assert fr.verify_market_cap(100, 1_000_000, 100_050_000, "USD") is True


def test_verify_market_cap_failure():
    out = capture_stdout(fr.verify_market_cap, price=100, shares=1_000_000,
                         reported=80_000_000, currency="USD")
    assert "❌ 警告" in out
    assert fr.verify_market_cap(100, 1_000_000, 80_000_000, "USD") is False


def test_cross_validate_tolerance():
    values = {"A": 100, "B": 102, "C": 108}
    # Median = 102, B is within 2%, C exceeds it

    result = fr.cross_validate("Test", values, unit="", tolerance_pct=2.0)
    assert result["all_consistent"] is False
    # Tight tolerance forces a failure

    result2 = fr.cross_validate("Test", values, unit="", tolerance_pct=0.5)
    assert result2["all_consistent"] is False


def test_benford_small_sample():
    # < 50 values => returns None and prints warning

    out = capture_stdout(fr.benford_check, list(range(1, 30)))
    assert "样本量不足" in out
    assert fr.benford_check(list(range(1, 30))) is None


def test_exact_calc_safe_expression():
    result = fr.exact_calc("10 * 1.5e2 + 3")
    assert result == 1503.0  # 10*150 + 3

Run the suite with pytest -q to confirm that core logic functions as documented across all validation paths.

Integration Testing via the CLI

Validate the command-line interface by invoking tools/financial_rigor.py using subprocess.run. This confirms that argument parsing, help messages, and sub-command dispatch function correctly.

import subprocess, json, pathlib

def test_cli_verify_market_cap():
    cmd = [
        "python3", "tools/financial_rigor.py",
        "verify-market-cap",
        "--price", "100",
        "--shares", "1000000",
        "--reported", "100050000",
        "--currency", "USD",
    ]
    result = subprocess.run(cmd, capture_output=True, text=True, check=True)
    assert "✅ 验证通过" in result.stdout

These integration tests ensure that the module behaves correctly when accessed as a standalone script rather than a library import.

Practical Effectiveness Checklist

Validate Decimal Arithmetic Accuracy

Compare calculations against hand-computed Decimal results to verify exact precision. For example, confirm that exact_calc("10 * 1.5e2 + 3") returns exactly 1503.0 without floating-point drift. Test verify_market_cap with values that produce exact multiples versus fractional shares to ensure the module handles decimal arithmetic correctly.

Verify Warning Thresholds

Feed data yielding deviations of 1%, 3%, and 6% to confirm the corresponding ✅/⚠️/❌ symbols appear correctly. For cross_validate, provide three sources with values [100, 101, 150] and a tolerance of 2%—the outlier (150) should be flagged while the consensus values pass.

Test Edge Cases and Error Handling

Ensure the module handles boundary conditions gracefully:

• Zero reported market cap: Verify the division guard prevents crashes.
• Small Benford samples: Confirm benford_check returns None and prints "样本量不足" for datasets with fewer than 50 values.
• Malformed expressions: Pass non-numeric strings to exact_calc and verify the "❌ 不安全的表达式" message appears without raising unhandled exceptions.
• Zero EPS: Verify that PE calculations handle zero earnings without raising errors.

Key Functions and File Paths

The effectiveness testing strategy covers these specific implementations in tools/financial_rigor.py:

• verify_market_cap: Validates that price × shares matches reported market cap within 5% deviation.
• cross_validate: Computes median consensus from multiple sources and flags deviations beyond the specified tolerance (default 2%).
• benford_check: Detects abnormal leading-digit distributions using MAD and chi-square statistics; requires 50+ samples.
• exact_calc: Safely evaluates arithmetic expressions with Decimal precision.
• three_scenario_valuation: Projects future EPS and target prices under optimistic, base, and pessimistic assumptions.

Additionally, tools/report_audit.py provides higher-level audit scripts that call financial_rigor functions during report generation, serving as a reference for real-world integration patterns.

Summary

• Test verify_market_cap with calculated values within and beyond the 5% deviation threshold to ensure boolean returns and correct status messages.
• Verify cross_validate flags sources exceeding the specified tolerance percentage, defaulting to 2%.
• Confirm benford_check returns None for samples under 50 values and detects anomalous leading-digit distributions in larger datasets.
• Use pytest to assert deterministic behavior of exact_calc and decimal-safe arithmetic operations.
• Validate CLI functionality via subprocess.run to ensure proper argument parsing and output formatting.

Frequently Asked Questions

What is the minimum sample size required for Benford’s law validation in financial_rigor.py?

The benford_check function requires at least 50 values to perform statistical analysis. If fewer samples are provided, the function returns None and prints a warning message indicating insufficient sample size.

How does the module handle decimal precision during calculations?

The module uses Python’s Decimal type for all arithmetic operations to avoid floating-point errors. Functions like exact_calc and verify_market_cap perform exact decimal calculations, ensuring that financial validations maintain precision critical for monetary values.

What are the default tolerance thresholds for data validation?

The verify_market_cap function flags deviations greater than 5% between calculated and reported market capitalization. For cross_validate, the default tolerance is 2%, flagging any source that deviates beyond this percentage from the median consensus.

Can I test the module without installing external dependencies?

Yes, the financial_rigor.py module is designed with zero dependencies. You only need pytest for running the test suite. The module can be imported directly or executed via CLI using standard Python libraries.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}