Using Python with uv for Zero-Setup DuckDB Queries on Remote Parquet Data

You can query remote Parquet files using DuckDB and Python with zero local setup by using uv run to automatically handle dependencies, as demonstrated in the BBL Tracker Public Database repository.

The nelsonjchen/bbl-tracker-public-db repository demonstrates a serverless architecture for analyzing Bambu Lab filament stock history using Python with uv for zero-setup DuckDB queries. The dataset consists of hourly Parquet shards stored on a public Cloudflare R2 bucket, indexed by a lightweight manifest.json file that enables selective downloading and querying without authentication.

How the BBL Tracker Public Database Works

Data Architecture: Parquet Shards on Cloudflare R2

The data layer resides entirely on a public R2 bucket at https://db-public.bbltracker.com. Files follow a deterministic naming convention (YYYY-MM-DD-HHMM.parquet) and contain hourly snapshots of product availability across different regions. This design eliminates the need for API keys or database credentials while keeping bandwidth usage minimal.

Manifest Discovery via JSON Index

A manifest.json file at the bucket root lists every available shard with timestamps and filenames. In script.py, the repository uses urllib.request to fetch this manifest and parse the JSON into a sorted list of files. This occurs in lines 13-21 of script.py, where the code constructs the URL, opens the connection, and loads the JSON response to identify available data ranges.

Zero-Setup Python Execution with uv

Automatic Dependency Resolution

uv is a fast, cross-platform Python package manager that resolves and installs dependencies on first run. The repository leverages this capability to eliminate manual environment setup. When you execute uv run script.py, uv automatically detects the required packages (duckdb, pandas) from inline metadata or imports, installs them into an isolated environment, and executes the script.

Running Queries Without pip install

The zero-setup workflow requires no pip install commands or requirements.txt files. As documented in the repository's README (lines 24-35), users simply install uv via the one-line installer, then run the example scripts. This approach ensures reproducibility across different machines while maintaining a minimal footprint.

Querying Remote Parquet Files with DuckDB

Manifest Parsing and File Selection

After downloading the manifest, script.py filters the file list to reduce network traffic. Lines 29-31 demonstrate selecting only the four most recent shards using recent_files = all_files[-4:], effectively limiting the query to the last 24 hours of data. Alternatively, reconstruct_db.py implements a 30-day cutoff by comparing timestamps against the current date (lines 34-38), allowing reconstruction of longer historical windows.

Constructing the read_parquet Query

DuckDB's read_parquet function accepts a list of URLs and streams the remote files directly into memory without full downloads. In script.py (starting at line 40), the code constructs a SQL query string that interpolates the selected URLs into the read_parquet([...]) function. This enables complex analytics across distributed files as if they were local tables.

Filtering and Aggregating Stock Data

The example query in script.py filters by region, aggregates availability statistics, and returns a Pandas DataFrame. DuckDB processes the SQL, reads only the necessary row groups from the remote Parquet files, and materializes the results. The script then prints the 15 items with the lowest in-stock percentages for the US region, highlighting potential supply bottlenecks like "Black PETG".

Building a Local Database for Offline Analysis

Reconstructing 30 Days of Data

For scenarios requiring repeated queries or offline access, reconstruct_db.py merges selected shards into a persistent local database. The script follows the same manifest discovery pattern but selects files from the past 30 days. It then constructs a CREATE TABLE ... AS SELECT * FROM read_parquet({urls}) statement (lines 60-66) to materialize all data into a single DuckDB file.

Exporting to Single DuckDB Files

The resulting bambu_stock.duckdb file contains the full 30-day history in a compact, columnar format. This file can be uploaded to AI code interpreter tools like ChatGPT's Code Interpreter or Claude's analysis features, enabling natural language queries against the structured data without requiring the original Python environment.

Summary

• The BBL Tracker Public Database stores hourly Parquet shards on Cloudflare R2, indexed by a public manifest.json file.
• Python with uv enables zero-setup execution by automatically installing duckdb and pandas when running uv run script.py.
• DuckDB queries remote Parquet files directly via read_parquet(), streaming only necessary data rather than full downloads.
• script.py demonstrates real-time bottleneck detection by analyzing the last 24 hours of stock data.
• reconstruct_db.py builds a local bambu_stock.duckdb file for offline analysis or AI tool integration.

Frequently Asked Questions

What is uv and why use it with DuckDB?

uv is a fast Python package manager and runner that eliminates manual environment setup. When analyzing data with DuckDB, uv handles dependency resolution automatically—running uv run script.py installs duckdb and pandas on first execution without requiring pip install or virtual environment management.

How does the BBL Tracker database handle large datasets without downloading everything?

The repository uses selective file discovery via manifest.json and DuckDB's remote Parquet streaming. Scripts like script.py select only the most recent 4 shards (24 hours) using all_files[-4:], while DuckDB's read_parquet() function reads only the necessary row groups from the remote URLs rather than downloading complete files.

Can I query the BBL Tracker data without keeping Python running continuously?

Yes. The reconstruct_db.py script creates a persistent local database file named bambu_stock.duckdb containing 30 days of historical data. This single file can be queried independently using any DuckDB client, or uploaded to AI code interpreters like ChatGPT for analysis without requiring the original Python scripts or internet access to the R2 bucket.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}