What Programming Languages and Libraries Are Used in ai-berkshire?

The ai-berkshire repository is a Python-centric investment research framework that leverages Playwright for web automation, standard library modules like argparse and decimal for CLI tools and precise financial calculations, and Bash scripts for deployment automation.

The ai-berkshire project, developed by xbtlin, is an open-source modular investment-research framework built on large-language-model agents. According to the source code analysis, the codebase implements a three-layer architecture where Python handles core logic, Playwright manages web scraping, and Bash scripts facilitate installation workflows.

Primary Programming Language (Python 3)

The foundation of ai-berkshire is Python 3, which powers the entire core logic and tool ecosystem. The repository implements a modular design where individual Python scripts serve as discrete Tools invoked by LLM agents. This architecture ensures that financial calculations, data gathering, and report generation remain reproducible and auditable.

Key Python Libraries and Dependencies

The project relies on a combination of standard library modules and third-party packages to handle financial calculations, web automation, and CLI interfaces.

Web Scraping and Browser Automation with Playwright

For data gathering from financial platforms like Snowball (Xueqiu), ai-berkshire uses Playwright. The tools/xueqiu_scraper.py script employs Playwright for both headful and headless browsing to fetch JSON data and scrape user timelines based on keyword filters.

# Example invocation from the CLI

python3 tools/xueqiu_scraper.py \
  --user-id 1247347556 \
  --keywords 拼多多,PDD,Temu,黄峥 \
  --output reports/拼多多/段永平雪球发言-PDD相关.md

Precise Financial Calculations using decimal

To avoid floating-point errors in financial arithmetic, the codebase extensively uses decimal.Decimal from the Python standard library. The tools/financial_rigor.py module performs exact arithmetic for market cap verification and cross-validation of financial metrics, ensuring investment calculations maintain precision.

# Running a financial-rigor verification

python3 tools/financial_rigor.py verify-market-cap \
  --price 510 --shares 9.11e9 --reported 4.65e12 --currency HKD

CLI and System Utilities

Several standard library modules support command-line interfaces and system operations:

• argparse: Powers CLI entry points in tools/financial_rigor.py and tools/report_audit.py
• asyncio: Handles asynchronous operations for concurrent agent processing
• pathlib, os, sys: File system and path management across the repository
• json, csv: Data serialization for watchlists and Morningstar fair value datasets
• datetime: Temporal calculations and report timestamps
• urllib, subprocess: Network requests and system command execution
• collections, random, re: Data structures and text processing utilities

Shell Scripting Components

Beyond Python, ai-berkshire utilizes Bash/POSIX shell scripts for automation and deployment. The scripts/install-codex-skills.sh script installs Codex skills, while tools/log-command.sh provides helper utilities for command logging and audit trails. These shell components bridge the gap between the Python tool ecosystem and system-level installation workflows.

Data Formats and Configuration Files

The framework uses JSON, CSV, and Markdown for data exchange and configuration:

• data/watchlist.json stores stock screening inputs for the screener tool
• data/morningstar_fair_value_20260519.csv contains fair value datasets for valuation analysis
• skills/investment-research.md defines agent skills in human-readable Markdown format

Architectural Implementation

According to the source code in assets/architecture.mmd and implementation files, the repository follows a Skill → Agent → Tool architecture:

1. Skills: Defined in Markdown files under skills/, exported via scripts/sync-codex-skills.py
2. Agents: Four parallel LLM agents that synthesize tool outputs into final reports
3. Tools: Python scripts performing precise calculations, external API calls, and data gathering

The scripts/sync-codex-skills.py script converts Markdown skill definitions into Codex-compatible packages, enabling seamless integration between documentation and executable agent capabilities.

Key Source Files and Their Implementations

Financial Verification Tool

tools/financial_rigor.py implements market cap verification using decimal.Decimal for exact arithmetic. This tool cross-validates reported financial metrics against calculated values to detect discrepancies in investment data.

Web Scraping Tool

tools/xueqiu_scraper.py integrates Playwright for scraping Xueqiu user timelines with keyword filtering. The script handles authentication, dynamic page loading, and structured data extraction from the Snowball platform.

Stock Screening Tool

tools/stock_screener.py processes JSON watchlists to generate filtered investment reports. It reads metadata from data/watchlist.json and outputs structured Markdown reports for agent consumption.

Skill Synchronization Script

scripts/sync-codex-skills.py automates the packaging of Markdown skill definitions into Codex-compatible formats. After editing skill files in the skills/ directory, running this script updates the agent capabilities.

Summary

• ai-berkshire is built primarily on Python 3 with a modular tool-based architecture separating Skills, Agents, and Tools
• Playwright enables reliable web scraping of financial data from platforms like Xueqiu (Snowball) in tools/xueqiu_scraper.py
• decimal.Decimal ensures precise financial calculations without floating-point errors, implemented in tools/financial_rigor.py
• Standard library modules (argparse, pathlib, json, datetime) handle CLI interfaces, file management, and data serialization
• Bash scripts in scripts/ automate Codex skill installation and deployment tasks
• The three-layer design (Skill → Agent → Tool) allows LLM agents to invoke Python tools while maintaining financial rigor and reproducibility

Frequently Asked Questions

Why does ai-berkshire use Playwright instead of requests for web scraping?

Playwright is used specifically in tools/xueqiu_scraper.py to handle dynamic JavaScript-rendered content on financial platforms like Xueqiu (Snowball). This allows for both headful and headless browsing capabilities that pure HTTP requests cannot replicate, ensuring access to modern web applications that require browser context.

How does ai-berkshire ensure accuracy in financial calculations?

The codebase utilizes Python's native decimal.Decimal class in tools/financial_rigor.py to perform exact decimal arithmetic. This eliminates floating-point precision errors that are unacceptable in financial analysis, market cap verification, and cross-validation of reported metrics against calculated values.

What is the purpose of the sync-codex-skills.py script?

scripts/sync-codex-skills.py processes Markdown files in the skills/ directory to generate Codex-compatible skill packages. This script bridges the gap between human-readable skill documentation (like skills/investment-research.md) and machine-executable agent capabilities, allowing researchers to define agent behaviors in Markdown while maintaining executable workflows.

Can ai-berkshire be extended with additional programming languages?

While the core framework is Python-centric, the modular architecture allows for Tool scripts in other languages as long as they adhere to the CLI interface expected by the Agent layer. However, the current implementation relies exclusively on Python for financial rigor (using decimal) and Playwright-based automation, making Python the practical choice for extensions requiring precision or web interaction.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}