# What Programming Languages and Libraries Are Used in ai-berkshire?

> Explore the programming languages and libraries powering ai-berkshire. Discover Python, Playwright, argparse, decimal, and Bash scripts for investment research and automation.

- Repository: [Xbt Lin/ai-berkshire](https://github.com/xbtlin/ai-berkshire)
- Tags: internals
- Published: 2026-07-04

---

**The ai-berkshire repository is a Python-centric investment research framework that leverages Playwright for web automation, standard library modules like `argparse` and `decimal` for CLI tools and precise financial calculations, and Bash scripts for deployment automation.**

The ai-berkshire project, developed by xbtlin, is an open-source modular investment-research framework built on large-language-model agents. According to the source code analysis, the codebase implements a three-layer architecture where Python handles core logic, Playwright manages web scraping, and Bash scripts facilitate installation workflows.

## Primary Programming Language (Python 3)

The foundation of ai-berkshire is **Python 3**, which powers the entire core logic and tool ecosystem. The repository implements a modular design where individual Python scripts serve as discrete **Tools** invoked by LLM agents. This architecture ensures that financial calculations, data gathering, and report generation remain reproducible and auditable.

## Key Python Libraries and Dependencies

The project relies on a combination of standard library modules and third-party packages to handle financial calculations, web automation, and CLI interfaces.

### Web Scraping and Browser Automation with Playwright

For data gathering from financial platforms like Snowball (Xueqiu), ai-berkshire uses **Playwright**. The [`tools/xueqiu_scraper.py`](https://github.com/xbtlin/ai-berkshire/blob/main/tools/xueqiu_scraper.py) script employs Playwright for both headful and headless browsing to fetch JSON data and scrape user timelines based on keyword filters.

```python

# Example invocation from the CLI

python3 tools/xueqiu_scraper.py \
  --user-id 1247347556 \
  --keywords 拼多多,PDD,Temu,黄峥 \
  --output reports/拼多多/段永平雪球发言-PDD相关.md

```

### Precise Financial Calculations using decimal

To avoid floating-point errors in financial arithmetic, the codebase extensively uses **`decimal.Decimal`** from the Python standard library. The [`tools/financial_rigor.py`](https://github.com/xbtlin/ai-berkshire/blob/main/tools/financial_rigor.py) module performs exact arithmetic for market cap verification and cross-validation of financial metrics, ensuring investment calculations maintain precision.

```python

# Running a financial-rigor verification

python3 tools/financial_rigor.py verify-market-cap \
  --price 510 --shares 9.11e9 --reported 4.65e12 --currency HKD

```

### CLI and System Utilities

Several standard library modules support command-line interfaces and system operations:

- **`argparse`**: Powers CLI entry points in [`tools/financial_rigor.py`](https://github.com/xbtlin/ai-berkshire/blob/main/tools/financial_rigor.py) and [`tools/report_audit.py`](https://github.com/xbtlin/ai-berkshire/blob/main/tools/report_audit.py)
- **`asyncio`**: Handles asynchronous operations for concurrent agent processing
- **`pathlib`**, **`os`**, **`sys`**: File system and path management across the repository
- **`json`**, **`csv`**: Data serialization for watchlists and Morningstar fair value datasets
- **`datetime`**: Temporal calculations and report timestamps
- **`urllib`**, **`subprocess`**: Network requests and system command execution
- **`collections`**, **`random`**, **`re`**: Data structures and text processing utilities

## Shell Scripting Components

Beyond Python, ai-berkshire utilizes **Bash/POSIX shell** scripts for automation and deployment. The [`scripts/install-codex-skills.sh`](https://github.com/xbtlin/ai-berkshire/blob/main/scripts/install-codex-skills.sh) script installs Codex skills, while [`tools/log-command.sh`](https://github.com/xbtlin/ai-berkshire/blob/main/tools/log-command.sh) provides helper utilities for command logging and audit trails. These shell components bridge the gap between the Python tool ecosystem and system-level installation workflows.

## Data Formats and Configuration Files

The framework uses **JSON**, **CSV**, and **Markdown** for data exchange and configuration:

- [`data/watchlist.json`](https://github.com/xbtlin/ai-berkshire/blob/main/data/watchlist.json) stores stock screening inputs for the screener tool
- `data/morningstar_fair_value_20260519.csv` contains fair value datasets for valuation analysis
- [`skills/investment-research.md`](https://github.com/xbtlin/ai-berkshire/blob/main/skills/investment-research.md) defines agent skills in human-readable Markdown format

## Architectural Implementation

According to the source code in `assets/architecture.mmd` and implementation files, the repository follows a **Skill → Agent → Tool** architecture:

1. **Skills**: Defined in Markdown files under `skills/`, exported via [`scripts/sync-codex-skills.py`](https://github.com/xbtlin/ai-berkshire/blob/main/scripts/sync-codex-skills.py)
2. **Agents**: Four parallel LLM agents that synthesize tool outputs into final reports
3. **Tools**: Python scripts performing precise calculations, external API calls, and data gathering

The [`scripts/sync-codex-skills.py`](https://github.com/xbtlin/ai-berkshire/blob/main/scripts/sync-codex-skills.py) script converts Markdown skill definitions into Codex-compatible packages, enabling seamless integration between documentation and executable agent capabilities.

## Key Source Files and Their Implementations

### Financial Verification Tool

[`tools/financial_rigor.py`](https://github.com/xbtlin/ai-berkshire/blob/main/tools/financial_rigor.py) implements market cap verification using `decimal.Decimal` for exact arithmetic. This tool cross-validates reported financial metrics against calculated values to detect discrepancies in investment data.

### Web Scraping Tool

[`tools/xueqiu_scraper.py`](https://github.com/xbtlin/ai-berkshire/blob/main/tools/xueqiu_scraper.py) integrates Playwright for scraping Xueqiu user timelines with keyword filtering. The script handles authentication, dynamic page loading, and structured data extraction from the Snowball platform.

### Stock Screening Tool

[`tools/stock_screener.py`](https://github.com/xbtlin/ai-berkshire/blob/main/tools/stock_screener.py) processes JSON watchlists to generate filtered investment reports. It reads metadata from [`data/watchlist.json`](https://github.com/xbtlin/ai-berkshire/blob/main/data/watchlist.json) and outputs structured Markdown reports for agent consumption.

### Skill Synchronization Script

[`scripts/sync-codex-skills.py`](https://github.com/xbtlin/ai-berkshire/blob/main/scripts/sync-codex-skills.py) automates the packaging of Markdown skill definitions into Codex-compatible formats. After editing skill files in the `skills/` directory, running this script updates the agent capabilities.

## Summary

- ai-berkshire is built primarily on **Python 3** with a modular tool-based architecture separating Skills, Agents, and Tools
- **Playwright** enables reliable web scraping of financial data from platforms like Xueqiu (Snowball) in [`tools/xueqiu_scraper.py`](https://github.com/xbtlin/ai-berkshire/blob/main/tools/xueqiu_scraper.py)
- **`decimal.Decimal`** ensures precise financial calculations without floating-point errors, implemented in [`tools/financial_rigor.py`](https://github.com/xbtlin/ai-berkshire/blob/main/tools/financial_rigor.py)
- **Standard library modules** (`argparse`, `pathlib`, `json`, `datetime`) handle CLI interfaces, file management, and data serialization
- **Bash scripts** in `scripts/` automate Codex skill installation and deployment tasks
- The three-layer design (Skill → Agent → Tool) allows LLM agents to invoke Python tools while maintaining financial rigor and reproducibility

## Frequently Asked Questions

### Why does ai-berkshire use Playwright instead of requests for web scraping?

Playwright is used specifically in [`tools/xueqiu_scraper.py`](https://github.com/xbtlin/ai-berkshire/blob/main/tools/xueqiu_scraper.py) to handle dynamic JavaScript-rendered content on financial platforms like Xueqiu (Snowball). This allows for both headful and headless browsing capabilities that pure HTTP requests cannot replicate, ensuring access to modern web applications that require browser context.

### How does ai-berkshire ensure accuracy in financial calculations?

The codebase utilizes Python's native `decimal.Decimal` class in [`tools/financial_rigor.py`](https://github.com/xbtlin/ai-berkshire/blob/main/tools/financial_rigor.py) to perform exact decimal arithmetic. This eliminates floating-point precision errors that are unacceptable in financial analysis, market cap verification, and cross-validation of reported metrics against calculated values.

### What is the purpose of the sync-codex-skills.py script?

[`scripts/sync-codex-skills.py`](https://github.com/xbtlin/ai-berkshire/blob/main/scripts/sync-codex-skills.py) processes Markdown files in the `skills/` directory to generate Codex-compatible skill packages. This script bridges the gap between human-readable skill documentation (like [`skills/investment-research.md`](https://github.com/xbtlin/ai-berkshire/blob/main/skills/investment-research.md)) and machine-executable agent capabilities, allowing researchers to define agent behaviors in Markdown while maintaining executable workflows.

### Can ai-berkshire be extended with additional programming languages?

While the core framework is Python-centric, the modular architecture allows for Tool scripts in other languages as long as they adhere to the CLI interface expected by the Agent layer. However, the current implementation relies exclusively on Python for financial rigor (using `decimal`) and Playwright-based automation, making Python the practical choice for extensions requiring precision or web interaction.