What Storage Engines Does pgrust Support? A Complete Technical Guide

pgrust supports seven storage engines—heap, unlogged (um), TOAST, temporary, WAL, Dynamic Shared Memory (DSM), and shared buffers—mirroring the complete PostgreSQL 13 storage stack in Rust.

pgrust is a faithful Rust re-implementation of PostgreSQL's core architecture. Its storage subsystem, located in crates/_support/types/types_storage, exposes the same storage managers as upstream PostgreSQL through identical abstractions like SMgrRelation and smgr hooks.

Overview of pgrust Storage Architecture

The pgrust storage layer replicates PostgreSQL's storage manager interface, exposing engines through the smgr abstraction layer defined in storage.rs and smgr.rs. Each engine serves distinct durability, performance, and memory requirements while maintaining compatibility with PostgreSQL's on-disk formats.

The Seven pgrust Storage Engines

Heap Storage Engine

The heap engine is the default storage manager for ordinary tables and indexes. It implements the on-disk page layout defined in PostgreSQL's heap_page.c and heapam.c equivalents. In pgrust, this logic resides in crates/_support/types/types_storage/src/storage.rs, where the SmgrRelation type dispatches read and write operations to the heap implementation.

Unlogged (UM) Storage

Unlogged tables use a variant of the heap manager that bypasses Write-Ahead Logging (WAL) for faster writes. The engine checks the relpersistence flag in RelFileLocator to determine logging behavior. While sharing the same physical layout as heap, unlogged tables trade durability for performance—data is lost after a crash.

TOAST (The Oversized-Attribute Storage Technique)

The TOAST engine automatically moves large column values exceeding page size limits out of the main heap into separate TOAST tables. This secondary storage manager operates transparently, storing oversized attributes in pg_toast relations while maintaining references in the main heap tuples.

Temporary Table Storage

Temporary tables utilize the heap storage engine but route data to per-session temporary tablespaces. The RelFileLocator structure identifies these as temporary relations, ensuring automatic cleanup when the session ends. The implementation reuses the standard heap code with special tablespace selection logic.

Write-Ahead Log (WAL)

The WAL engine manages the pg_xlog files for crash recovery and replication. Defined in wal.rs (part of storage.rs), it provides XLogInsert and XLogRecPtr types for atomic record writing. Unlike user-facing storage engines, WAL operates as a background logging manager recording changes for all other engines.

Dynamic Shared Memory (DSM)

DSM provides on-demand shared memory segments attachable by any backend process. Used for inter-process communication and structures like dsa (Dynamic Shared Area), this engine manages segments via dsm_handle and dsa_handle types defined in storage.rs.

Shared Buffer Manager

The shared buffers engine implements the buffer-pool manager (bufmgr) that caches pages from any storage engine in shared memory. It provides the Buffer and BufferIsValid abstractions along with associated lock structures, ensuring consistent page access across concurrent connections.

Working with Storage Engines in Code

Opening a Heap Relation

use pgrust::types_storage::{
    RelFileLocator, Relation, Buffer, ReadBufferMode,
    smgr::SmgrRelation, XLogRecPtr,
};

// Build a RelFileLocator for a regular table (heap)
let locator = RelFileLocator::new(spc_oid, db_oid, rel_filenumber);

// Open the relation via the SMgr interface
let mut rel = SmgrRelation::open(&locator)?;

// Read a block from the heap (page 0) into a buffer
let buf = rel.read_buffer(0, ReadBufferMode::Normal)?;
let page_data = buf.get_page_data();

Creating Unlogged Tables

use pgrust::types_storage::{Relation, RelFileLocator};

// Set the `relpersistence` flag to `U` (unlogged) when creating the relation
let locator = RelFileLocator::new(spc_oid, db_oid, rel_filenumber);
let mut rel = Relation::create_unlogged(&locator)?;

Writing to WAL

use pgrust::types_storage::{XLogInsert, XLogRecPtr};

// Start a WAL insert operation
let mut wal = XLogInsert::new();
wal.insert_record(/* record data */);
let lsn: XLogRecPtr = wal.flush();

Allocating DSM Segments

use pgrust::types_storage::{dsm_handle, dsm_create, dsm_detach};

let dsm: dsm_handle = dsm_create(/* size */)?;
/* use the segment … */
dsm_detach(dsm);

Key Implementation Files

The storage engines are implemented across these critical source files:

• crates/_support/types/types_storage/src/storage.rs: Core storage definitions including buffers, locks, WAL structures, DSM APIs, and relation locators.
• crates/_support/types/types_storage/src/smgr.rs: Storage manager abstraction that dispatches operations to the appropriate engine (heap, um, toast).

Summary

• pgrust implements seven storage engines that mirror PostgreSQL 13's architecture
• Heap handles standard tables and indexes with full durability
• Unlogged (UM) provides heap-like performance without WAL overhead for transient data
• TOAST transparently manages oversized attributes outside main heap pages
• Temporary tables use heap storage with automatic session-scoped cleanup
• WAL provides crash recovery infrastructure for all storage operations
• DSM enables dynamic shared memory allocation for inter-process communication
• Shared buffers cache pages across all engines for concurrent access

Frequently Asked Questions

How do pgrust storage engines compare to PostgreSQL's?

pgrust storage engines maintain byte-for-byte compatibility with PostgreSQL 13's on-disk formats. The SMgrRelation and smgr hook abstractions match PostgreSQL's C implementation exactly, allowing pgrust to read and write PostgreSQL data files directly.

What is the difference between heap and unlogged storage in pgrust?

Both use identical page layouts in storage.rs, but unlogged tables set the relpersistence flag to U in RelFileLocator, causing the WAL engine to skip logging their modifications. This provides faster writes but eliminates crash recovery guarantees for that data.

When should I use Dynamic Shared Memory (DSM) in pgrust?

Use DSM when you need to share memory segments between multiple backend processes, such as for parallel query coordination or maintaining shared state across connections. The dsm_create and dsm_attach functions in storage.rs manage these segments.

Does pgrust support custom storage engines like PostgreSQL extensions?

Currently, pgrust supports the standard PostgreSQL storage stack (heap, toast, etc.) as implemented in types_storage. The smgr abstraction layer exists to support extensibility, but custom storage engines would require modifications to the dispatch logic in smgr.rs.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}