How the pgrust Query Optimizer Works: A Deep Dive into the Rust Implementation

The pgrust query optimizer re-implements PostgreSQL's planner in safe Rust using a modular crate architecture that mirrors the original C codebase, processing queries through stages like variable flattening, catalog lookups, path generation, and join ordering.

The pgrust project provides a faithful Rust rewrite of PostgreSQL's backend, including its sophisticated query optimization engine. Understanding how the pgrust query optimizer operates reveals how safe Rust patterns can replicate complex C-based database internals while maintaining the same performance characteristics and extensibility hooks.

Entry Point and the Planner Hook

When a query arrives at the pgrust server, the optimization process begins at the planner hook installed in the traffic cop layer. In crates/backend/tcop/postgres/src/simple_query.rs, the pgss_planner function serves as the entry point that delegates to the Rust implementation:

pub fn pgss_planner<'mcx>(… ) -> … {
    planner_seams::standard_planner::call(mcx, parse, query_string,…)
}

This hook mirrors PostgreSQL's standard_planner interface but routes execution into the Rust optimizer arena. After parsing produces a Query AST, the hook forwards the structure to the pipeline of optimizer crates.

The Optimizer Pipeline Architecture

The pgrust query optimizer follows PostgreSQL's classic planning stages, with each phase implemented as a separate crate under the backend/optimizer/util hierarchy. This seam-based architecture allows each component to be unit-tested in isolation while cooperating through safe Rust boundaries.

Variable Flattening and Normalization

The first stage processes Var nodes to flatten and normalize variable references. The var.rs file in crates/backend/optimizer/util/vars/src/var.rs handles operations like flatten_group_exprs, ensuring that column references are resolved correctly before cost estimation begins.

Target-List and Relation Management

Next, tlist.rs constructs PathTarget objects that represent the query's projection list, while relnode.rs (located at crates/backend/optimizer/util/relnode/src/lib.rs) computes relid sets—dependency bitmaps that track which base relations each node references. These structures enable the optimizer to understand table relationships and projection requirements.

Catalog Statistics and Metadata

The plancat crate (crates/backend/optimizer/util/plancat/src/lib.rs) performs catalog lookups to fetch physical statistics. Functions like get_rel_data_width and get_typavgwidth retrieve table sizes, column widths, and index statistics from the system catalogs, feeding this data into the cost model.

Clause Simplification and Cost Estimation

The clauses crate (crates/backend/optimizer/util/clauses/src/lib.rs) implements constant folding, selectivity estimation, and the cost model itself. It exposes functions like eq_sel and range_sel for predicate selectivity, and reads GUC variables (Grand Unified Configuration) such as cpu_operator_cost, seq_page_cost, and random_page_cost from crates/backend/utils/misc/guc_tables/src/vars.rs:

pub static cpu_operator_cost: GucRealVar = GucSlot::new("cpu_operator_cost");

These variables determine the relative costs of CPU operations versus sequential or random I/O, allowing the optimizer to compare different execution strategies quantitatively.

Path Generation and Access Methods

The pathnode crate (crates/backend/optimizer/util/pathnode/src/lib.rs) enumerates viable access paths for each relation. It generates path objects representing sequential scans, index scans, bitmap index scans, and other access methods. Each path carries a cost computed using the GUC-defined constants, enabling the optimizer to compare execution strategies.

Join Ordering and Path Selection

Join planning occurs in the joininfo crate (crates/backend/optimizer/util/joininfo/src/lib.rs), which re-implements PostgreSQL's joinsearch.c logic in idiomatic Rust. The generate_join_paths function builds a graph of RestrictInfo objects representing join predicates, then explores possible join orders:

pub fn generate_join_paths(root: &mut PlannerInfo, …) { … }

Using the cost model from the clauses crate, the join optimizer evaluates different tree shapes (left-deep, bushy, etc.) and selects the cheapest plan. The winning paths from each relation are then assembled into the final Plan tree.

Extensibility Through the Seam Model

Pgrust preserves PostgreSQL's extensibility through a seam architecture that mirrors the original hook system. Each optimizer component declares an "owner seam" (e.g., backend-optimizer-util-var) that exposes a safe Rust boundary. Developers can add new seam crates (like backend-optimizer-util-pathnode-seams) to override or augment behavior without modifying core logic, enabling custom cost models or access methods while maintaining memory safety.

Practical Usage and Configuration

You can control the pgrust query optimizer through GUC variables exposed in the client API. The following example demonstrates enabling sequential scans and adjusting CPU costs:

use pgrust::client::Client;
use pgrust::utils::guc_tables::vars::{enable_seqscan, cpu_operator_cost};

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let mut client = Client::connect("host=localhost dbname=test", None)?;

    // Configure optimizer behavior
    enable_seqscan::set(true);
    cpu_operator_cost::set(0.001);

    let rows = client.query(
        "SELECT * FROM orders o JOIN customers c ON o.cust_id = c.id WHERE o.amount > 100",
        &[],
    )?;

    println!("Returned {} rows", rows.len());
    Ok(())
}

To inspect planner decisions, enable debug logging via log_planner_stats:

use pgrust::utils::guc_tables::vars::log_planner_stats;

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let mut client = Client::connect("host=localhost dbname=test", None)?;
    log_planner_stats::set(true);
    
    client.query("EXPLAIN (VERBOSE) SELECT * FROM orders", &[])?;
    Ok(())
}

Summary

• The pgrust query optimizer entry point is pgss_planner in simple_query.rs, which calls into the Rust planner via planner_seams::standard_planner::call.
• Optimization proceeds through specialized crates: vars (variable handling), relnode (relation sets), plancat (catalog stats), clauses (cost estimation), pathnode (access paths), and joininfo (join ordering).
• Cost estimation relies on GUC variables like cpu_operator_cost and seq_page_cost exposed through guc_tables.
• The seam architecture allows safe extension of optimizer components without modifying core code.
• All planning occurs within the optimizer arena, a memory context that mirrors PostgreSQL's PlannerInfo lifecycle for deterministic resource management.

Frequently Asked Questions

How does pgrust maintain compatibility with PostgreSQL's optimizer?

Pgrust preserves PostgreSQL's query optimizer behavior by re-implementing the same algorithms and data structures (like RestrictInfo and PathTarget) in safe Rust, using the original C code as a reference for the joinsearch.c and clauses.c logic. The project maintains identical GUC variable names and cost constants, ensuring that query plans match those produced by standard PostgreSQL.

Can I extend the pgrust optimizer with custom planning logic?

Yes, through the seam architecture. Each optimizer component exposes a seam interface (e.g., backend-optimizer-util-var) that you can implement in a new crate to override specific behaviors. This allows injection of custom cost models or access methods while maintaining Rust's memory safety guarantees and without forking the core codebase.

Where does the pgrust optimizer store intermediate planning data?

Intermediate data lives in the optimizer arena, a dedicated memory context that mirrors PostgreSQL's PlannerInfo structure. This arena manages the lifecycle of Path objects, JoinInfo structures, and temporary calculations, ensuring that all allocations are deterministic and properly freed when planning completes, preventing memory leaks during complex query optimization.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}