How to Implement Benford's Law Detection for Financial Data Fraud Analysis

The benford_check() function in tools/financial_rigor.py compares the leading-digit distribution of a numeric series against Benford's expected logarithmic frequencies to flag potential fraud, returning a boolean conformity flag and statistical metrics.

The xbtlin/ai-berkshire repository provides a production-ready implementation of Benford's law for financial statement auditing. By analyzing the first digits of quantitative disclosures—such as revenue, net income, or EPS—analysts can spot anomalous patterns that deviate from the natural 30.1% / 17.6% / … distribution. This article explains the architecture of the built-in benford_check utility and demonstrates how to integrate it into standalone scripts and automated report pipelines.

Architecture of the Detection Module

The fraud detection logic lives entirely inside the financial rigor utilities, making it reusable across CLI audit tools and interactive notebooks.

The benford_check Function Core

In [tools/financial_rigor.py at line 227](https://github.com/xbtlin/ai-berkshire/blob/main/tools/financial_rigor.py#L227), the function is defined as:

def benford_check(values: list) -> tuple:
    ...

The implementation performs three discrete steps:

1. Digit Extraction – It iterates over the input list, converts each number to a string, and captures the first non‑zero digit (1‑9).
2. Frequency Calculation – It builds a histogram of the nine possible digits and normalises the counts to percentages.
3. Statistical Testing – It compares the observed distribution against the theoretical Benford probabilities (1:30.1%, 2:17.6%, …, 9:4.6%) using a chi‑square test, returning the statistic and a boolean flag indicating conformity.

Integration with Cross‑Validation

The same file invokes the detector at line 451 inside the cross_validate workflow. When the audit engine processes a batch of extracted financial figures, it automatically feeds each numeric series to benford_check; non‑conforming results trigger a warning in the final validation report.

How to Use Benford's Law Detection

Stand‑Alone Analysis

Import the utility directly to sanity‑check a single array of numbers:

from tools.financial_rigor import benford_check

# Example: annual revenues in millions

revenues = [
    12.4, 9.8, 15.2, 22.5, 18.0,
    31.7, 45.9, 28.3, 39.1, 50.6,
]

conforms, chi2, observed, expected = benford_check(revenues)

print(f"Conforms to Benford: {conforms}")
print(f"Chi‑square: {chi2:.3f}")

The function returns a bool (True when the data follow Benford's law), the chi‑square statistic, and two dictionaries (observed, expected) that map digits 1‑9 to their respective percentages.

Automating Report Audits

The [tools/report_audit.py](https://github.com/xbtlin/ai-berkshire/blob/main/tools/report_audit.py) module automatically extracts numeric tables from Markdown disclosures. You can replicate that pipeline manually:

from tools.report_audit import extract_data_points
from tools.financial_rigor import benford_check

with open("reports/2024_Q1_report.md", "r", encoding="utf-8") as f:
    md_text = f.read()

points = extract_data_points(md_text)          # returns (label, value, unit) tuples

values = [float(v) for (_, v, _) in points if "Revenue" in _.lower()]

ok, chi2, _, _ = benford_check(values)

if not ok:
    print("⚠️  Revenue series deviates from Benford’s law — flag for review")

Visualizing Digit Distributions

To generate bar charts for stakeholder reports, unpack the frequency dictionaries returned by benford_check:

import matplotlib.pyplot as plt
from tools.financial_rigor import benford_check

values = [...]  # your financial series

_, _, observed, expected = benford_check(values)

digits = range(1, 10)
plt.bar([d - 0.2 for d in digits],
        [observed[d] for d in digits],
        width=0.4, label="Observed")
plt.bar([d + 0.2 for d in digits],
        [expected[d] for d in digits],
        width=0.4, label="Benford Expected")
plt.xticks(digits)
plt.xlabel("Leading digit")
plt.ylabel("Frequency (%)")
plt.legend()
plt.show()

Key Files and Source Locations

• tools/financial_rigor.py – Core utilities for quantitative validation; contains the benford_check definition (line 227) and its invocation inside the cross‑validation helper (line 451). View source
• tools/report_audit.py – Parses Markdown reports, extracts numeric data points, and feeds them to the fraud detector. View source
• tests/test_financial_rigor.py – Unit tests verifying the statistical correctness of benford_check against known distributions. View source
• skills/financial-data.md – Documentation for the “financial‑data” skill describing available fraud‑detection helpers, including Benford's law. View source

Summary

• The benford_check function in tools/financial_rigor.py (line 227) implements the full statistical test—digit extraction, frequency normalization, and chi‑square comparison.
• It returns a conformity boolean, the chi‑square statistic, and observed/expected distributions suitable for plotting or logging.
• tools/report_audit.py automatically invokes the check for every numeric series extracted from Markdown financial reports, enabling zero‑config fraud screening.
• Extensive unit tests in tests/test_financial_rigor.py ensure the algorithm matches theoretical Benford probabilities within acceptable tolerance.

Frequently Asked Questions

What is Benford's law and why does it detect financial fraud?

Benford's law states that in naturally occurring datasets, the digit 1 appears as the leading digit about 30 % of the time, with higher digits appearing with decreasing frequency following a logarithmic distribution. Fraudulent or manipulated data tend to have uniform or artificially biased first‑digit patterns, causing the chi‑square statistic to exceed the conformity threshold.

What data types can I pass to benford_check?

The function accepts any iterable of numeric values (integers or floats). It internally casts non‑zero numbers to strings to isolate the first digit, so negative numbers and scientific notation are handled safely; zero values are ignored because they have no leading digit.

How does the implementation handle small sample sizes?

The current implementation relies on the chi‑square test, which becomes unreliable when the sample size is below roughly 50–100 observations. For smaller series, the function still returns the observed distribution, but you should treat the conformity flag as advisory and consider supplementing with domain‑specific heuristics.

Can I replace the chi‑square test with a Kolmogorov–Smirnov test?

Yes. The source code in tools/financial_rigor.py isolates the statistical comparison in a dedicated internal helper. You can fork the repository and swap the chi‑square calculation for a KS‑test or add an optional method="ks" parameter without altering the digit‑extraction logic.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}