How to Implement Benford's Law Detection for Financial Data Authenticity in Python

The AI-Berkshire repository provides a zero-dependency benford_check function in tools/financial_rigor.py that analyzes financial datasets for digit-distribution anomalies using Mean Absolute Deviation (MAD) and chi-square testing against Benford's expected frequencies.

The AI-Berkshire toolkit offers a lightweight command-line utility for validating financial data authenticity through statistical rigor. The benford_check implementation performs first-digit analysis to detect potential data manipulation without requiring external dependencies beyond the Python standard library.

Core Implementation in tools/financial_rigor.py

The Benford's Law detection engine resides entirely within tools/financial_rigor.py, implementing a classical first-digit test through logarithmic scaling and statistical hypothesis testing.

First-Digit Extraction Logic

The algorithm isolates the most significant digit (1–9) from each numeric value using logarithmic mathematics. As implemented in lines 22–28 of financial_rigor.py, the routine applies math.log10 and math.floor to determine the leading digit without string conversion:

# Conceptual excerpt from lines 22-28

import math

def extract_leading_digit(value):
    if value > 0:
        return int(10 ** (math.log10(abs(value)) % 1))
    return None

This approach ensures efficient numeric processing suitable for large financial datasets.

Expected Distribution Constants

The module pre-computes Benford's theoretical probabilities in the _BENFORD dictionary (lines 11–13), storing the expected frequency for each digit $d$ (1 through 9) as $\log_{10}(1 + 1/d)$:

_BENFORD = {d: math.log10(1 + 1/d) for d in range(1, 10)}

This constant provides the baseline against which observed distributions are compared.

Statistical Validation Metrics

The benford_check function calculates two primary statistical measures (lines 41–56) to quantify divergence from expected distributions:

• Mean Absolute Deviation (MAD) – The average absolute difference between observed and expected digit frequencies. This metric drives the conformity classification.
• Chi-square ($\chi^2$) – Evaluates the goodness-of-fit between observed and expected distributions.

The implementation applies standard thresholds to categorize results:

• MAD < 0.006: "Close" conformity
• MAD < 0.012: "Acceptable" conformity
• MAD < 0.015: "Marginally Acceptable"
• MAD ≥ 0.015: "Nonconforming"

Result Structure and Return Value

The function returns a structured dictionary (line 81) designed for programmatic consumption:

{
    "mad": float,           # Mean Absolute Deviation

    "chi2": float,          # Chi-square statistic

    "conformity": str,      # Classification level

    "is_conforming": bool   # True if MAD < 0.015

}

Additionally, the CLI interface (lines 62–78) renders a formatted table displaying observed versus expected frequencies with deviation flags.

Command-Line Usage

The financial_rigor.py module exposes Benford's Law detection through a dedicated CLI sub-command. The parser configuration (lines 64–78) accepts a JSON array of values and forwards them to benford_check:

python3 tools/financial_rigor.py benford \
    --values '[1234, 2345, 3456, 4567, 5678, 6789, 7890, 8910, 9123]'

The command outputs a formatted comparison table and a final pass/fail conformity message based on the MAD threshold evaluation.

Programmatic Integration

Basic Function Call

Import benford_check directly to analyze financial series within Python applications:

import json
from tools.financial_rigor import benford_check

# Example: quarterly revenue figures

values = json.loads('[1200, 950, 870, 1020, 1110, 945, 1300, 1150, 980, 1050]')

result = benford_check(values)

print(f"MAD: {result['mad']:.4f}")
print(f"Conforms to Benford: {result['is_conforming']}")

Pipeline Integration with Cross-Validation

The function integrates with the toolkit's cross_validate utility to create multi-stage validation pipelines:

from tools.financial_rigor import benford_check, cross_validate

# Load financial figures from multiple sources

source_vals = {
    "AnnualReport": 1_050_000,
    "YahooFinance": 1_045_200,
    "Bloomberg": 1_048_500
}

# First, cross-validate the raw numbers

cv = cross_validate("Revenue", source_vals, unit="千元")
if cv["all_consistent"]:
    # Then run Benford on the combined set

    combined = list(source_vals.values())
    benford = benford_check(combined)
    if not benford["is_conforming"]:
        print("⚠️ Potential data manipulation detected!")

Key Architectural Details

Aspect	Implementation Detail
Dependencies	Python standard library only (math module)
Digit Range	1–9 (excludes 0 per Benford's Law)
CLI Entry Point	parser.add_subparser("benford", ...) invoking benford_check(values)
Statistical Thresholds	Hardcoded MAD limits: 0.006, 0.012, 0.015
Output Format	Structured dict for API use; formatted table for CLI

Summary

• The benford_check function in tools/financial_rigor.py provides zero-dependency first-digit analysis for financial data validation.
• MAD thresholds (0.006/0.012/0.015) determine conformity classifications ranging from "Close" to "Nonconforming".
• The implementation uses logarithmic scaling (math.log10, math.floor) for efficient digit extraction without string parsing overhead.
• Results are available both as a formatted CLI table and as a machine-readable dictionary ({mad, chi2, conformity, is_conforming}).
• The tool integrates seamlessly with the toolkit's existing cross_validate function for multi-stage financial validation pipelines.

Frequently Asked Questions

What is Benford's Law and why does it apply to financial data?

Benford's Law states that in naturally occurring datasets, the leading digit $d$ occurs with probability $\log_{10}(1 + 1/d)$. Authentic financial data typically follows this distribution, while fabricated or manipulated data often deviates significantly. The AI-Berkshire implementation uses this statistical property to flag potential anomalies without requiring external benchmarks.

How does the Mean Absolute Deviation (MAD) threshold work?

The MAD quantifies the average absolute difference between observed and expected digit frequencies. According to the financial_rigor.py implementation, values below 0.006 indicate "Close" conformity, below 0.012 indicate "Acceptable" conformity, and below 0.015 indicate "Marginally Acceptable" conformity. Any MAD above 0.015 triggers a "Nonconforming" classification and sets is_conforming to False.

Can the benford_check function handle negative numbers or zeros?

The implementation processes absolute values (abs(value)) and specifically excludes zero from the first-digit analysis, as Benford's Law applies only to digits 1 through 9. The digit extraction logic in lines 22–28 filters out non-positive values automatically, ensuring statistical validity.

Is this method suitable for all types of financial datasets?

Benford's Law applies best to datasets spanning multiple orders of magnitude without artificial minimum/maximum boundaries. The ai-berkshire tool works optimally for raw transaction amounts, revenue figures, or expense reports. However, assigned identifiers ( invoice numbers), uniform distributions (fixed prices), or small sample sizes (<300 observations) may produce false positives regardless of data authenticity.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}