# How to Run Tests for AI Berkshire: Validating the Research Codebase

> Discover how to run tests for AI Berkshire by executing audit scripts and validation commands. Ensure research outputs and data integrity are sound.

- Repository: [Xbt Lin/ai-berkshire](https://github.com/xbtlin/ai-berkshire)
- Tags: testing
- Published: 2026-07-09

---

**AI Berkshire does not ship a traditional unit-test suite; instead, you run tests by executing the audit scripts and tool validation commands that verify research outputs and data integrity.**

The xbtlin/ai-berkshire repository organizes its validation workflow around research integrity rather than conventional software testing. Because this is a research-oriented codebase, you won't find a `pytest` or `unittest` directory. Instead, learning how to run tests for AI Berkshire means leveraging the built-in **report-audit** scripts and **Codex skill synchronization** tools that function as the project's test harness.

## Understanding the Testing Architecture

Unlike standard Python libraries, AI Berkshire treats validation as a research audit process. The codebase lacks a dedicated `tests/` folder or `pytest` configuration. Instead, validation occurs through executable scripts that check report formatting, citation integrity, and data consistency. According to the repository structure, the primary validation logic resides in [`tools/report_audit.py`](https://github.com/xbtlin/ai-berkshire/blob/main/tools/report_audit.py), which serves as the main test harness for verifying generated research outputs.

## Step-by-Step Guide to Running Tests

Follow these four steps to validate the AI Berkshire codebase and ensure all components function correctly.

### Install the Development Environment

Before running any validation commands, install the Python dependencies required by the project. The repository typically lists these in [`requirements.txt`](https://github.com/xbtlin/ai-berkshire/blob/main/requirements.txt) or [`pyproject.toml`](https://github.com/xbtlin/ai-berkshire/blob/main/pyproject.toml).

```bash
pip install -r requirements.txt

```

Ensure the repository's helper scripts remain executable and that your Python environment includes all financial analysis dependencies referenced by the tool modules.

### Execute the Report Audit Tool

The [`tools/report_audit.py`](https://github.com/xbtlin/ai-berkshire/blob/main/tools/report_audit.py) script functions as the primary test suite for the project. This script walks through the `reports/` directory and validates formatting, citations, and data consistency issues.

```bash
python -m tools.report_audit

```

When you run this command, it scans generated reports for structural errors and integrity problems, effectively serving as the comprehensive test harness for research outputs.

### Synchronize and Validate Codex Skills

After modifying any skill markdown files in the `skills/` directory, you must validate the changes by regenerating the Codex skill files. The [`scripts/sync-codex-skills.py`](https://github.com/xbtlin/ai-berkshire/blob/main/scripts/sync-codex-skills.py) script performs this synchronization and checks for syntax errors, exiting with a non-zero status if validation fails.

```bash
python3 scripts/sync-codex-skills.py

```

This step acts as a critical test for the skill generation pipeline, ensuring that markdown source files in `skills/*.md` translate correctly into functional Codex skills without syntax errors.

### Run Individual Tool Scripts for Sanity Checks

Many analytical utilities contain built-in self-checking logic that raises exceptions on malformed data. Running these scripts with sample inputs serves as ad-hoc testing. For example, you can validate the financial rigor calculations by importing `calculate_rigour` from [`tools/financial_rigor.py`](https://github.com/xbtlin/ai-berkshire/blob/main/tools/financial_rigor.py).

```python
from tools.financial_rigor import calculate_rigour
print(calculate_rigour([100, 105, 110]))

```

Similarly, the momentum backtest tools ([`tools/momentum_backtest.py`](https://github.com/xbtlin/ai-berkshire/blob/main/tools/momentum_backtest.py) and [`tools/momentum_backtest_v2.py`](https://github.com/xbtlin/ai-berkshire/blob/main/tools/momentum_backtest_v2.py)) perform internal validation and raise errors if input data lacks required integrity.

## Key Files for Test Validation

Understanding these specific file paths helps you navigate the validation process:

- **[`tools/report_audit.py`](https://github.com/xbtlin/ai-berkshire/blob/main/tools/report_audit.py)** - Walks through `reports/` and verifies format, citations, and data consistency; this is the primary test-like validator.
- **[`scripts/sync-codex-skills.py`](https://github.com/xbtlin/ai-berkshire/blob/main/scripts/sync-codex-skills.py)** - Regenerates Codex skill files from `skills/*.md` and checks for syntax errors; failures act as test failures.
- **[`tools/financial_rigor.py`](https://github.com/xbtlin/ai-berkshire/blob/main/tools/financial_rigor.py)** - Contains reusable financial calculations including the `calculate_rigour` function with built-in sanity checks.
- **[`tools/momentum_backtest.py`](https://github.com/xbtlin/ai-berkshire/blob/main/tools/momentum_backtest.py)** and **[`tools/momentum_backtest_v2.py`](https://github.com/xbtlin/ai-berkshire/blob/main/tools/momentum_backtest_v2.py)** - Perform momentum-based backtests and raise errors if input data is malformed.
- **`skills/`** - Directory containing source markdown files for Claude Code and Codex skills; changes here must be validated by the sync script.

## Adding Formal Unit Tests

If you require a traditional testing framework, you can extend the repository by creating standard `pytest` tests that import the tool modules and assert on their return values. Create a `tests/` directory and write Python files that import functions from `tools/financial_rigor` or other modules until such a suite becomes part of the main repository.

## Summary

- **AI Berkshire** uses audit scripts rather than traditional unit tests for validation.
- Run **`python -m tools.report_audit`** to verify report integrity and formatting.
- Execute **`python3 scripts/sync-codex-skills.py`** after editing files in `skills/` to validate Codex skill generation.
- Use individual tool scripts like **[`tools/financial_rigor.py`](https://github.com/xbtlin/ai-berkshire/blob/main/tools/financial_rigor.py)** for ad-hoc validation of specific calculations.
- The repository validates functionality through research output integrity rather than software assertion libraries.

## Frequently Asked Questions

### Is there a pytest test suite for AI Berkshire?

No. The xbtlin/ai-berkshire repository does not ship with `pytest` or `unittest` files. Instead, the project relies on the [`tools/report_audit.py`](https://github.com/xbtlin/ai-berkshire/blob/main/tools/report_audit.py) script and skill synchronization commands to validate functionality and data integrity.

### What does the report_audit.py script check?

The [`tools/report_audit.py`](https://github.com/xbtlin/ai-berkshire/blob/main/tools/report_audit.py) script walks through the `reports/` directory and verifies formatting standards, citation integrity, and data consistency. It functions as the primary test harness by scanning generated research outputs for structural errors and validation failures.

### How do I validate changes to skill files?

After modifying any markdown files in the `skills/` directory, run `python3 scripts/sync-codex-skills.py`. This script regenerates the Codex skill files and validates that the syntax is correct, exiting with a non-zero status code if it encounters errors in the source markdown.

### Can I add formal unit tests to the repository?

Yes. While AI Berkshire currently lacks a formal test suite, you can create one by adding a `tests/` directory and writing `pytest` scripts that import functions from modules like [`tools/financial_rigor.py`](https://github.com/xbtlin/ai-berkshire/blob/main/tools/financial_rigor.py) and assert on their return values, such as testing the `calculate_rigour` function with known inputs.