# How to Test the Effectiveness of the financial_rigor.py Module in ai-berkshire

> Learn how to test the effectiveness of the financial_rigor.py module in ai-berkshire. Implement unit tests, verify CLI integration, and handle edge cases for robust validation.

- Repository: [Xbt Lin/ai-berkshire](https://github.com/xbtlin/ai-berkshire)
- Tags: testing
- Published: 2026-07-04

---

**To test the effectiveness of the [`financial_rigor.py`](https://github.com/xbtlin/ai-berkshire/blob/main/financial_rigor.py) module, implement deterministic unit tests for each validation function—such as `verify_market_cap`, `cross_validate`, and `benford_check`—asserting both successful validations and proper threshold warnings, then verify CLI integration and edge-case handling.**

The [`financial_rigor.py`](https://github.com/xbtlin/ai-berkshire/blob/main/financial_rigor.py) module in the xbtlin/ai-berkshire repository provides a zero-dependency Python toolkit for validating financial data using exact decimal arithmetic. Testing its effectiveness ensures that market-cap verifications, valuation metrics, and Benford’s law checks return deterministic, accurate results under various conditions.

## Unit Testing Strategy for financial_rigor.py

### Import and Direct Function Testing

Import the module directly using `from tools import financial_rigor as fr` to bypass the CLI wrapper and test functions in isolation. Supply deterministic input data with known expected outputs—for example, verify that `verify_market_cap` returns `True` when calculated price × shares falls within **5%** of the reported value, and `False` when deviation exceeds this threshold.

Assert on return values to validate both success and failure paths. For `cross_validate`, provide multiple data sources and verify that the function correctly identifies outliers when they exceed the specified tolerance percentage (default **2%**). Capture `stdout` using `io.StringIO` to verify that user-facing messages such as "✅ 验证通过" or "❌ 警告" appear correctly.

### Sample pytest Implementation

Create a test file in a `tests/` directory to validate core functionality:

```python

# tests/test_financial_rigor.py

import json
import io
import sys

from tools import financial_rigor as fr


def capture_stdout(func, *args, **kwargs):
    """Utility to capture printed output."""
    captured = io.StringIO()
    sys_stdout = sys.stdout
    sys.stdout = captured
    try:
        func(*args, **kwargs)
    finally:
        sys.stdout = sys_stdout
    return captured.getvalue()


def test_verify_market_cap_success():
    # price * shares = 100 * 1_000_000 = 100_000_000

    out = capture_stdout(fr.verify_market_cap, price=100, shares=1_000_000,
                         reported=100_050_000, currency="USD")
    assert "✅ 验证通过" in out
    assert fr.verify_market_cap(100, 1_000_000, 100_050_000, "USD") is True


def test_verify_market_cap_failure():
    out = capture_stdout(fr.verify_market_cap, price=100, shares=1_000_000,
                         reported=80_000_000, currency="USD")
    assert "❌ 警告" in out
    assert fr.verify_market_cap(100, 1_000_000, 80_000_000, "USD") is False


def test_cross_validate_tolerance():
    values = {"A": 100, "B": 102, "C": 108}
    # Median = 102, B is within 2%, C exceeds it

    result = fr.cross_validate("Test", values, unit="", tolerance_pct=2.0)
    assert result["all_consistent"] is False
    # Tight tolerance forces a failure

    result2 = fr.cross_validate("Test", values, unit="", tolerance_pct=0.5)
    assert result2["all_consistent"] is False


def test_benford_small_sample():
    # < 50 values => returns None and prints warning

    out = capture_stdout(fr.benford_check, list(range(1, 30)))
    assert "样本量不足" in out
    assert fr.benford_check(list(range(1, 30))) is None


def test_exact_calc_safe_expression():
    result = fr.exact_calc("10 * 1.5e2 + 3")
    assert result == 1503.0  # 10*150 + 3

```

Run the suite with `pytest -q` to confirm that core logic functions as documented across all validation paths.

## Integration Testing via the CLI

Validate the command-line interface by invoking [`tools/financial_rigor.py`](https://github.com/xbtlin/ai-berkshire/blob/main/tools/financial_rigor.py) using `subprocess.run`. This confirms that **argument parsing**, **help messages**, and **sub-command dispatch** function correctly.

```python
import subprocess, json, pathlib

def test_cli_verify_market_cap():
    cmd = [
        "python3", "tools/financial_rigor.py",
        "verify-market-cap",
        "--price", "100",
        "--shares", "1000000",
        "--reported", "100050000",
        "--currency", "USD",
    ]
    result = subprocess.run(cmd, capture_output=True, text=True, check=True)
    assert "✅ 验证通过" in result.stdout

```

These integration tests ensure that the module behaves correctly when accessed as a standalone script rather than a library import.

## Practical Effectiveness Checklist

### Validate Decimal Arithmetic Accuracy

Compare calculations against hand-computed `Decimal` results to verify exact precision. For example, confirm that `exact_calc("10 * 1.5e2 + 3")` returns exactly `1503.0` without floating-point drift. Test `verify_market_cap` with values that produce exact multiples versus fractional shares to ensure the module handles decimal arithmetic correctly.

### Verify Warning Thresholds

Feed data yielding deviations of **1%**, **3%**, and **6%** to confirm the corresponding ✅/⚠️/❌ symbols appear correctly. For `cross_validate`, provide three sources with values `[100, 101, 150]` and a tolerance of **2%**—the outlier (150) should be flagged while the consensus values pass.

### Test Edge Cases and Error Handling

Ensure the module handles boundary conditions gracefully:

- **Zero reported market cap**: Verify the division guard prevents crashes.
- **Small Benford samples**: Confirm `benford_check` returns `None` and prints "样本量不足" for datasets with fewer than **50** values.
- **Malformed expressions**: Pass non-numeric strings to `exact_calc` and verify the "❌ 不安全的表达式" message appears without raising unhandled exceptions.
- **Zero EPS**: Verify that PE calculations handle zero earnings without raising errors.

## Key Functions and File Paths

The effectiveness testing strategy covers these specific implementations in [`tools/financial_rigor.py`](https://github.com/xbtlin/ai-berkshire/blob/main/tools/financial_rigor.py):

- **`verify_market_cap`**: Validates that `price × shares` matches reported market cap within **5%** deviation.
- **`cross_validate`**: Computes median consensus from multiple sources and flags deviations beyond the specified tolerance (default **2%**).
- **`benford_check`**: Detects abnormal leading-digit distributions using MAD and chi-square statistics; requires **50+** samples.
- **`exact_calc`**: Safely evaluates arithmetic expressions with `Decimal` precision.
- **`three_scenario_valuation`**: Projects future EPS and target prices under optimistic, base, and pessimistic assumptions.

Additionally, [`tools/report_audit.py`](https://github.com/xbtlin/ai-berkshire/blob/main/tools/report_audit.py) provides higher-level audit scripts that call `financial_rigor` functions during report generation, serving as a reference for real-world integration patterns.

## Summary

- Test `verify_market_cap` with calculated values within and beyond the **5%** deviation threshold to ensure boolean returns and correct status messages.
- Verify `cross_validate` flags sources exceeding the specified tolerance percentage, defaulting to **2%**.
- Confirm `benford_check` returns `None` for samples under **50** values and detects anomalous leading-digit distributions in larger datasets.
- Use `pytest` to assert deterministic behavior of `exact_calc` and decimal-safe arithmetic operations.
- Validate CLI functionality via `subprocess.run` to ensure proper argument parsing and output formatting.

## Frequently Asked Questions

### What is the minimum sample size required for Benford’s law validation in financial_rigor.py?

The `benford_check` function requires at least **50 values** to perform statistical analysis. If fewer samples are provided, the function returns `None` and prints a warning message indicating insufficient sample size.

### How does the module handle decimal precision during calculations?

The module uses Python’s `Decimal` type for all arithmetic operations to avoid floating-point errors. Functions like `exact_calc` and `verify_market_cap` perform exact decimal calculations, ensuring that financial validations maintain precision critical for monetary values.

### What are the default tolerance thresholds for data validation?

The `verify_market_cap` function flags deviations greater than **5%** between calculated and reported market capitalization. For `cross_validate`, the default tolerance is **2%**, flagging any source that deviates beyond this percentage from the median consensus.

### Can I test the module without installing external dependencies?

Yes, the [`financial_rigor.py`](https://github.com/xbtlin/ai-berkshire/blob/main/financial_rigor.py) module is designed with zero dependencies. You only need `pytest` for running the test suite. The module can be imported directly or executed via CLI using standard Python libraries.