# BiomeJS Performance Benchmarks: A Technical Deep Dive into the Criterion-Based Testing Architecture

> Explore BiomeJS performance benchmarks with this technical deep dive. Discover how Criterion tests parser, formatter, and analyzer throughput, reporting bytes-per-second metrics.

- Repository: [Biome/biome](https://github.com/biomejs/biome)
- Tags: performance
- Published: 2026-06-19

---

**BiomeJS performance benchmarks use Criterion to measure parser, formatter, and analyzer throughput on real-world JavaScript libraries, caching inputs under `target/` and reporting bytes-per-second metrics.**

The `biomejs/biome` repository ships with a comprehensive suite of micro-benchmarks designed to validate the performance of its JavaScript and TypeScript tooling. These benchmarks exercise the core stages of the pipeline—parsing, formatting, and analysis—using real-world library code downloaded on-the-fly. Understanding the **BiomeJS performance benchmarks** architecture helps contributors optimize the toolchain and gives users confidence in the engine's throughput characteristics.

## Benchmark Architecture and Pipeline Stages

The Biome codebase organizes benchmarks as **Criterion** test suites spread across multiple crates. Each major stage of the JavaScript processing pipeline has its own dedicated benchmark driver:

### Parsing Benchmarks

The JavaScript and TypeScript parser benchmarks live in [`crates/biome_js_parser/benches/js_parser.rs`](https://github.com/biomejs/biome/blob/main/crates/biome_js_parser/benches/js_parser.rs). This file defines two measurement modes:

- **Uncached parsing**: Measures `biome_js_parser::parse` with cold inputs
- **Cached parsing**: Measures `parse_js_with_cache` for repeated analysis scenarios

Each input file becomes a separate `BenchmarkId`, with throughput set to the byte size of the source to enable **bytes/s** reporting.

### Formatting Benchmarks

Located at [`crates/biome_js_formatter/benches/js_formatter.rs`](https://github.com/biomejs/biome/blob/main/crates/biome_js_formatter/benches/js_formatter.rs), the formatter benchmarks parse the input once, then repeatedly invoke `biome_js_formatter::format_node` on the resulting AST. This isolates formatting latency from parsing overhead and measures the actual layout engine performance.

### Analysis Benchmarks

The static analysis pipeline is benchmarked in [`crates/biome_js_analyze/benches/js_analyzer.rs`](https://github.com/biomejs/biome/blob/main/crates/biome_js_analyze/benches/js_analyzer.rs). Following the same pattern as the parser and formatter, this driver exercises the `biome_js_analyze::analyze` function against real-world codebases to detect performance regressions in lint rules and semantic analysis.

## The BenchCase Helper and Input Management

All benchmarks rely on the **BenchCase** type defined in [`crates/biome_test_utils/src/bench_case.rs`](https://github.com/biomejs/biome/blob/main/crates/biome_test_utils/src/bench_case.rs). This helper handles the logistics of benchmark input:

1. **URL Resolution**: Each benchmark suite points to a text file (e.g., [`libs-js.txt`](https://github.com/biomejs/biome/blob/main/libs-js.txt), [`libs-ts.txt`](https://github.com/biomejs/biome/blob/main/libs-ts.txt)) containing URLs of popular npm packages hosted on GitHub
2. **Deterministic Caching**: For each URL, the helper resolves a deterministic filename, downloads the content via `ureq`, and stores it under `target/<hash>.js`
3. **Isolation**: This caching strategy guarantees repeatable runs while isolating network latency from the measured numbers

The `BenchCase` exposes methods like `code()` and `filename()` that benchmarks use to access the source text and metadata.

## Memory Allocation and Statistical Rigor

To eliminate allocator noise on large inputs, the benchmark binaries configure a fast global allocator using `#[cfg]` guards:

- **Windows**: `mimalloc`
- **Linux/macOS**: `jemalloc`

This ensures that memory management overhead does not skew the results when processing large JavaScript bundles. Criterion groups are created per stage (e.g., `"js_parser"`), with each input file receiving its own `BenchmarkId` and throughput measurement based on file size.

## Running Benchmarks Locally

The benchmarks are ordinary Rust binaries generated by Cargo. After cloning the repository, build all benchmarks without running them:

```bash
cargo bench --all --no-run

```

To execute a specific benchmark suite:

```bash

# Parse JavaScript and TypeScript sources

cargo bench --bench js_parser

# Format the parsed ASTs

cargo bench --bench js_formatter

# Run the analyzer pipeline

cargo bench --bench js_analyzer

```

For CI integration or more stable local numbers, tune Criterion through environment variables:

```bash

# Increase sample size for statistical stability

CRITERION_SAMPLE_SIZE=30 cargo bench --bench js_parser

```

The output displays time, mean, standard deviation, and throughput (bytes per second). Criterion also emits JSON reports under `target/criterion` for automated tracking.

## Adding Custom Benchmarks

To measure a new language feature or specific code pattern, extend the existing infrastructure. First, add a URL to the appropriate suite file (e.g., [`crates/biome_js_parser/benches/libs-js.txt`](https://github.com/biomejs/biome/blob/main/crates/biome_js_parser/benches/libs-js.txt)):

```text
https://raw.githubusercontent.com/some/package/v1.2.3/src/optional_chaining.js

```

Then create a minimal benchmark harness:

```rust
use biome_js_parser::{parse, JsParserOptions};
use biome_languages::JsFileSource;
use biome_test_utils::BenchCase;
use criterion::{Criterion, BenchmarkId, Throughput, black_box, criterion_group, criterion_main};

fn bench_custom_parser(c: &mut Criterion) {
    let case = BenchCase::try_from(
        "https://raw.githubusercontent.com/some/package/v1.2.3/src/optional_chaining.js"
    ).expect("download case");
    let code = case.code();
    let src = JsFileSource::default();

    let mut group = c.benchmark_group("custom_js_parser");
    group.throughput(Throughput::Bytes(code.len() as u64));
    group.bench_with_input(
        BenchmarkId::new("optional_chaining", case.filename()),
        &code,
        |b, _| b.iter(|| black_box(parse(code, src, JsParserOptions::default()))),
    );
    group.finish();
}

criterion_group!(custom_parser, bench_custom_parser);
criterion_main!(custom_parser);

```

Run the new benchmark with:

```bash
cargo bench --bench custom_parser

```

## Summary

- **BiomeJS performance benchmarks** are organized as Criterion test suites across `biome_js_parser`, `biome_js_formatter`, and `biome_js_analyze` crates
- The **BenchCase** helper in [`crates/biome_test_utils/src/bench_case.rs`](https://github.com/biomejs/biome/blob/main/crates/biome_test_utils/src/bench_case.rs) manages real-world library downloads and caching under `target/`
- Benchmarks use platform-specific fast allocators (`mimalloc` on Windows, `jemalloc` on Unix) to eliminate memory management noise
- Throughput is measured in **bytes per second**, providing realistic metrics for JavaScript processing speed
- Execute specific suites with `cargo bench --bench <name>` and tune stability via `CRITERION_SAMPLE_SIZE`

## Frequently Asked Questions

### How does BiomeJS ensure benchmark results are reproducible?

The **BenchCase** type downloads source files from URLs listed in suite files like [`libs-js.txt`](https://github.com/biomejs/biome/blob/main/libs-js.txt), then caches them under `target/<hash>.js` using deterministic filenames. This caching strategy ensures that network latency does not affect measurements and that subsequent runs process identical inputs.

### What allocator does Biome use for benchmarking?

The benchmark binaries configure a fast global allocator using conditional compilation. On Windows, they use **mimalloc**; on Linux and macOS, they use **jemalloc**. This prevents standard allocator overhead from skewing performance measurements on large JavaScript files.

### Can I benchmark Biome against my own JavaScript code?

Yes. You can add your own URLs to the library list files (e.g., [`libs-js.txt`](https://github.com/biomejs/biome/blob/main/libs-js.txt)) or create a custom benchmark harness that uses `BenchCase::try_from()` to fetch your code. The `BenchCase` API provides `code()` and `filename()` methods to integrate your sources into Criterion benchmarks.

### Where does Biome store benchmark results?

Criterion outputs results to the console as formatted tables showing time, mean, standard deviation, and throughput. Additionally, JSON reports are emitted under `target/criterion` for programmatic analysis and CI integration.