# Write-Ahead Logging (WAL) Architecture in pgrust: A Rust Reimplementation of PostgreSQL's Transaction Log

> Explore pgrust's Write-Ahead Logging WAL architecture rebuilt in safe Rust. Discover record assembly, compression, and crash recovery with PostgreSQL binary compatibility. Learn more.

- Repository: [Michael Malis/pgrust](https://github.com/malisper/pgrust)
- Tags: internals
- Published: 2026-07-13

---

**pgrust implements PostgreSQL's WAL system in safe Rust through the `wal` and `xloginsert` crates, providing record assembly, compression, and crash recovery while maintaining binary compatibility with PostgreSQL's transaction log format.**

The pgrust project reimplements PostgreSQL's core engine in safe Rust, including its Write-Ahead Logging (WAL) architecture that ensures ACID compliance and crash recovery. This Rust-based WAL system mirrors the original C implementation's semantics while leveraging Rust's memory safety guarantees. The architecture spans multiple crates, with the core logic residing in `wal` for data types and `xloginsert` for record construction.

## Core WAL Components and Crate Structure

The WAL implementation is distributed across two primary crates that handle distinct responsibilities:

- **`wal` crate** (`crates/_support/types/wal/`): Defines the **WAL record vocabulary**, constants, and decoded structures like `XLogRecord` and `DecodedXLogRecord` in [`wal/src/wal.rs`](https://github.com/malisper/pgrust/blob/main/wal/src/wal.rs) and [`wal/src/lib.rs`](https://github.com/malisper/pgrust/blob/main/wal/src/lib.rs).
- **`xloginsert` crate** (`crates/backend/access/transam/xloginsert/`): Provides the **WAL insertion API** including `XLogBeginInsert`, `XLogRegisterBuffer`, and `XLogInsert` in [`xloginsert/src/lib.rs`](https://github.com/malisper/pgrust/blob/main/xloginsert/src/lib.rs).
- **Seam layer**: The `transam_xlog` crate exposes the low-level `xlog_insert_record` seam that writes assembled records to WAL segment files.

## WAL Record Vocabulary and Header Structures

The foundation of pgrust's WAL system lies in its strict adherence to PostgreSQL's binary format, defined in [`crates/_support/types/wal/src/wal.rs`](https://github.com/malisper/pgrust/blob/main/crates/_support/types/wal/src/wal.rs).

### Resource Manager IDs and Flags

The crate defines standard resource-manager IDs and flag constants that match PostgreSQL's `access/xlog*` headers:

```rust
/// Resource‑manager IDs
pub const RM_XLOG_ID:  RmgrId = 0;
pub const RM_XACT_ID:  RmgrId = 1;
pub const RM_SMGR_ID:  RmgrId = 2;

/// Flags used in `xl_info`
pub const XLR_INFO_MASK: uint8 = 0x0F;
pub const XLR_SPECIAL_REL_UPDATE: uint8 = 0x01;
pub const XLR_CHECK_CONSISTENCY:   uint8 = 0x02;

```

### The XLogRecord Header

Every WAL record begins with the `XLogRecord` header struct, defined in [`crates/_support/types/wal/src/lib.rs`](https://github.com/malisper/pgrust/blob/main/crates/_support/types/wal/src/lib.rs), which stores metadata including length, transaction ID, and CRC:

```rust
#[derive(Clone, Copy, Debug, Eq, PartialEq)]
pub struct XLogRecord {
    xl_tot_len: uint32,
    xl_xid:    TransactionId,
    xl_prev:   XLogRecPtr,
    xl_info:   uint8,
    xl_rmid:   RmgrId,
    xl_crc:    pg_crc32c,
}

```

During recovery, the system uses `DecodedXLogRecord`, which contains the header plus the main data slice and an array of `DecodedBkpBlock` structs describing block references, images, and per-block data.

## The WAL Insertion Pipeline

The `xloginsert` crate implements a five-step insertion workflow that matches PostgreSQL's C implementation exactly. This process is orchestrated through thread-local state managed in `XLogInsertState`.

### Step 1-3: Initialization and Buffer Registration

Record construction begins with `XLogBeginInsert()`, which verifies WAL insertion is allowed and activates the thread-local state. Buffer registration follows through `XLogRegisterBuffer(block_id, buffer, flags)`, which copies page images:

```rust
// 1️⃣ Start a new WAL record
XLogBeginInsert()?;

// 2️⃣ Register the modified page (force a full‑page image)
XLogRegisterBuffer(0, buffer, REGBUF_FORCE_IMAGE)?;

// 3️⃣ Attach any extra payload
XLogRegisterData(data)?;

```

The `alloc_block()` function performs the copy, storing the page inside `RegBuf.page` along with the relation tag, fork number, and block number. Developers can also attach per-block data using `XLogRegisterBufData(block_id, data)`.

### Step 4-5: Assembly and Insertion

Before insertion, `XLogSetRecordFlags(flags)` sets metadata flags like `XLOG_INCLUDE_ORIGIN`. The `XLogInsert(rmid, info)` function then triggers `XLogRecordAssemble`, which performs several critical operations:

- **Full-page decision**: `XLogCheckBufferNeedsBackup` determines if a full-page image is required based on the current redo pointer and the page's LSN.
- **Hole removal**: For standard pages, the code computes holes using the `lower`/`upper` values from the page header.
- **Compression**: Optionally compresses images using PGLZ via `XLogCompressBackupBlock`.
- **CRC calculation**: Computes CRC32C over the header (minus the CRC field) and all data spans.
- **Size validation**: Enforces the `XLOG_RECORD_MAX_SIZE` limit of approximately 1 GB.

The assembled record passes to `xlog_insert_record` in the `transam_xlog_seams` module, which atomically writes the spans to WAL segment files while holding the WAL insertion lock.

## Recovery and Redo Processing

During crash recovery, the **redo functions** consume `DecodedXLogRecord` structures. The decoder provides PostgreSQL-compatible helper methods like `info()`, `xid()`, `has_block_ref()`, and `block_image_apply()`.

Resource managers receive a `RedoRecord` view containing the info byte, main data slice, and block presence flags. This abstraction, defined in [`crates/_support/types/wal/src/lib.rs`](https://github.com/malisper/pgrust/blob/main/crates/_support/types/wal/src/lib.rs), allows redo functions to apply changes without managing low-level memory unsafe operations.

## Practical WAL Insertion Example

The following example demonstrates inserting a custom WAL record for resource manager `RM_GENERIC_ID`:

```rust
use pgrust::backend::access::transam::xloginsert::*;
use pgrust::wal::{RM_GENERIC_ID, XLR_SPECIAL_REL_UPDATE};

fn log_my_event(buffer: Buffer, data: &[u8]) -> PgResult<XLogRecPtr> {
    // 1️⃣ Start a new WAL record
    XLogBeginInsert()?;

    // 2️⃣ Register the modified page (force a full‑page image)
    XLogRegisterBuffer(0, buffer, REGBUF_FORCE_IMAGE)?;

    // 3️⃣ Attach any extra payload
    XLogRegisterData(data)?;

    // 4️⃣ Set a flag (e.g., include replication origin)
    XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN);

    // 5️⃣ Insert the record – RM = generic, info = special‑rel‑update
    XLogInsert(RM_GENERIC_ID, XLR_SPECIAL_REL_UPDATE)
}

```

For recovery, iterate over decoded blocks:

```rust
use pgrust::wal::{DecodedXLogRecord, DecodedBkpBlock};

fn process_record(rec: &DecodedXLogRecord) {
    // Access header fields
    let xid = rec.xid();
    let info = rec.info();

    // Iterate over block references
    for (i, blk) in rec.blocks().iter().enumerate() {
        if blk.has_image() && blk.apply_image() {
            // Restore the page image...
        }
        if let Some(data) = blk.data() {
            // Apply per‑block data...
        }
    }

    // Main data payload
    let payload = rec.data();
    // …handle payload…
}

```

## Summary

- **Safe Rust implementation**: pgrust replicates PostgreSQL's WAL semantics in safe Rust, eliminating memory safety vulnerabilities while preserving binary compatibility.
- **Modular crate structure**: The `wal` crate handles data types and constants, while `xloginsert` manages record construction and assembly.
- **Five-step insertion**: The `XLogBeginInsert` → `XLogRegisterBuffer` → `XLogRegisterData` → `XLogSetRecordFlags` → `XLogInsert` pipeline mirrors PostgreSQL's exact workflow.
- **Compression and validation**: Records undergo PGLZ compression (via `XLogCompressBackupBlock`) and CRC32C validation during assembly.
- **Recovery support**: The `DecodedXLogRecord` and `RedoRecord` abstractions provide type-safe access to WAL data during crash recovery.

## Frequently Asked Questions

### How does pgrust maintain binary compatibility with PostgreSQL's WAL format?

pgrust defines identical constants, struct layouts, and bit flags to PostgreSQL's C headers in [`crates/_support/types/wal/src/wal.rs`](https://github.com/malisper/pgrust/blob/main/crates/_support/types/wal/src/wal.rs). The `XLogRecord` struct uses the same field ordering and sizes as the C implementation, ensuring that WAL files produced by pgrust are readable by PostgreSQL and vice versa.

### What is the maximum size of a single WAL record in pgrust?

The `XLogRecordAssemble` function enforces a hard limit of `XLOG_RECORD_MAX_SIZE` (approximately 1 GB) during record construction. If the assembled record exceeds this size, the insertion returns an error before reaching the `xlog_insert_record` seam.

### How does pgrust handle full-page writes during WAL insertion?

The `XLogCheckBufferNeedsBackup` function determines whether a full-page image is required based on the current redo pointer and the page's LSN. When `REGBUF_FORCE_IMAGE` is passed to `XLogRegisterBuffer`, or when the page LSN indicates it was modified before the last checkpoint, the system copies the entire page image and optionally compresses it using PGLZ.

### Which compression algorithm does pgrust use for WAL full-page images?

According to the `xloginsert` implementation in [`crates/backend/access/transam/xloginsert/src/lib.rs`](https://github.com/malisper/pgrust/blob/main/crates/backend/access/transam/xloginsert/src/lib.rs), pgrust uses **PGLZ** (PostgreSQL's LZ-based compression) via the `XLogCompressBackupBlock` function. This matches PostgreSQL's default compression method for full-page images.