# How pgrust Implements Write-Ahead Logging (WAL): A Rust Port of PostgreSQL's Architecture

> Explore how pgrust implements Write-Ahead Logging by porting PostgreSQL's WAL to Rust. Discover the WAL crate and xloginsert module for efficient record handling.

- Repository: [Michael Malis/pgrust](https://github.com/malisper/pgrust)
- Tags: internals
- Published: 2026-07-13

---

**pgrust implements Write-Ahead Logging by porting PostgreSQL's C-based WAL machinery to safe Rust, organizing the core logic into the `wal` crate for record definitions and the `xloginsert` module for the five-step insertion pipeline that assembles, compresses, and writes records to disk.**

The `malisper/pgrust` project provides a Rust-native reimplementation of PostgreSQL's storage engine, including a faithful port of its Write-Ahead Logging (WAL) system. This implementation preserves PostgreSQL's exact semantics and recovery guarantees while leveraging Rust's memory safety, with the core WAL vocabulary, record structures, and insertion logic distributed across the `wal` and `xloginsert` crates.

## WAL Record Architecture and Core Types

The foundation of pgrust's WAL system resides in the `wal` crate, which defines the constants, headers, and decoded structures necessary for both record creation and replay.

### The XLogRecord Header Structure

In [`crates/_support/types/wal/src/wal.rs`](https://github.com/malisper/pgrust/blob/main/crates/_support/types/wal/src/wal.rs), pgrust defines the `XLogRecord` header that precedes every WAL entry on disk. This struct mirrors PostgreSQL's binary layout exactly:

```rust
#[derive(Clone, Copy, Debug, Eq, PartialEq)]
pub struct XLogRecord {
    xl_tot_len: uint32,
    xl_xid:    TransactionId,
    xl_prev:   XLogRecPtr,
    xl_info:   uint8,
    xl_rmid:   RmgrId,
    xl_crc:    pg_crc32c,
}

```

The file also exports the resource-manager ID constants (e.g., `RM_XLOG_ID`, `RM_XACT_ID`, `RM_SMGR_ID`) and flag bits (`XLR_INFO_MASK`, `XLR_SPECIAL_REL_UPDATE`, `XLR_CHECK_CONSISTENCY`) that identify which subsystem owns the record and how it should be processed.

### Decoded Record Structures for Recovery

For replay operations, [`crates/_support/types/wal/src/lib.rs`](https://github.com/malisper/pgrust/blob/main/crates/_support/types/wal/src/lib.rs) provides `DecodedXLogRecord`, which unpacks the header and provides convenient accessors for the main data payload and block references. Each block reference is represented as a `DecodedBkpBlock` containing the relation tag, fork number, block number, and optional full-page image or data slice. The `RedoRecord` view type exposes a simplified interface specifically for resource-manager redo functions, offering methods like `info()`, `xid()`, `record_origin()`, and `has_block_ref()` that correspond to PostgreSQL's `XLogRecGet*` helpers.

## The WAL Insertion Pipeline

The `xloginsert` crate ([`crates/backend/access/transam/xloginsert/src/lib.rs`](https://github.com/malisper/pgrust/blob/main/crates/backend/access/transam/xloginsert/src/lib.rs)) implements the procedural API for constructing and emitting WAL records, following PostgreSQL's five-step pattern exactly.

### Step-by-Step Record Construction

Pgrust constructs a WAL record through a strict sequence of API calls that manage thread-local insertion state:

1. **Start a new record** – `XLogBeginInsert()` verifies that WAL insertion is allowed and marks the thread-local `XLogInsertState` as active.

2. **Register buffers** – `XLogRegisterBuffer(block_id, buffer, flags)` copies the page image from a `Buffer` and stores the relation metadata. The copy is performed in `alloc_block()` and stored inside `RegBuf.page`.

3. **Add arbitrary data** – `XLogRegisterData(data)` appends bytes to the main chunk, while `XLogRegisterBufData(block_id, data)` attaches data to a specific block reference.

4. **Set record flags** – `XLogSetRecordFlags(flags)` (e.g., `XLOG_INCLUDE_ORIGIN`) modifies the `xl_info` field.

5. **Insert the record** – `XLogInsert(rmid, info)` finalizes the operation by calling `XLogRecordAssemble` and handing the assembled spans to the low-level `xlog_insert_record` seam.

### Record Assembly and Compression

The `XLogRecordAssemble` function in [`xloginsert/src/lib.rs`](https://github.com/malisper/pgrust/blob/main/xloginsert/src/lib.rs) performs the heavy lifting of converting registered buffers and data into the on-disk format:

- **Full-page decision** – `XLogCheckBufferNeedsBackup` compares the page's LSN against the current redo pointer to determine if a full-page image is required.
- **Hole removal** – For standard pages, the code calculates the hole between `lower` and `upper` in the page header and excludes it from the image.
- **Compression** – `XLogCompressBackupBlock` compresses the image using PGLZ (PostgreSQL's LZ compression) when beneficial.
- **CRC calculation** – The implementation computes a CRC32C accumulator over the header (excluding the CRC field itself) and every data span, matching PostgreSQL's `COMP_CRC32C` logic.
- **Size guarding** – If the assembled record exceeds `XLOG_RECORD_MAX_SIZE` (approximately 1 GB), the function returns an error before writing.

### Writing to WAL Segment Files

Once assembled, `XLogInsert` obtains the current full-page-write flag and redo pointer via `transam_xlog_seams::get_full_page_write_info`. It then calls `xlog_insert_record` (implemented in the `transam_xlog` crate) which receives the slice of byte spans, the full-page-write LSN, flags, and the number of full-page images. This seam writes the data atomically while holding the WAL insertion lock, returning the end LSN upon completion. The insertion loop handles retry logic for full-page-write conflicts before clearing state with `XLogResetInsertion`.

## Recovery and Redo Processing

During crash recovery, pgrust uses the `DecodedXLogRecord` type to iterate over WAL entries. The decoder provides methods to inspect block images (`block_image_apply()`), access per-block data, and read the main payload. Resource managers receive a `RedoRecord` view generated from the decoded record, allowing them to reconstruct page state by applying the saved images or replaying the logical operations encoded in the record data.

## Summary

- **pgrust's WAL implementation** mirrors PostgreSQL's C architecture but is written in safe Rust, located primarily in the `wal` and `xloginsert` crates.
- **Record headers** are defined by `XLogRecord` in [`wal/src/wal.rs`](https://github.com/malisper/pgrust/blob/main/wal/src/wal.rs), while decoded structures like `DecodedXLogRecord` and `DecodedBkpBlock` live in [`wal/src/lib.rs`](https://github.com/malisper/pgrust/blob/main/wal/src/lib.rs).
- **Insertion follows a five-step pattern**: begin, register buffers, register data, set flags, and insert via `XLogInsert` in [`xloginsert/src/lib.rs`](https://github.com/malisper/pgrust/blob/main/xloginsert/src/lib.rs).
- **Assembly handles** full-page image decisions, hole removal, PGLZ compression via `XLogCompressBackupBlock`, and CRC32C calculation.
- **Actual disk writes** occur through the `xlog_insert_record` seam in the `transam_xlog` crate, which manages segment file I/O and atomicity.

## Frequently Asked Questions

### How does pgrust's WAL implementation differ from PostgreSQL's original C code?

While pgrust preserves the exact binary format, semantics, and recovery guarantees of PostgreSQL's WAL, it reimplements the logic in safe Rust. The core types like `XLogRecord` and `DecodedXLogRecord` use Rust's struct layout and memory safety guarantees instead of C pointers and manual memory management, but the file organization in `crates/_support/types/wal/` and `crates/backend/access/transam/xloginsert/` directly corresponds to PostgreSQL's `access/xlog*` source files.

### What is the maximum size of a single WAL record in pgrust?

Pgrust enforces the same limit as PostgreSQL: `XLOG_RECORD_MAX_SIZE`, which is approximately 1 GB. The `XLogRecordAssemble` function checks the assembled record size against this constant and returns an error if the limit is exceeded, preventing unbounded memory usage during record construction.

### How does pgrust calculate CRC values for WAL records?

The implementation uses CRC32C (Castagnoli) calculated over the entire record header (excluding the `xl_crc` field itself) and all attached data spans. This occurs in `XLogRecordAssemble` within [`xloginsert/src/lib.rs`](https://github.com/malisper/pgrust/blob/main/xloginsert/src/lib.rs), using the same accumulator logic as PostgreSQL's `COMP_CRC32C` macro to ensure compatibility with existing PostgreSQL WAL files.

### Which compression algorithm does pgrust use for full-page images?

Pgrust currently uses PGLZ (PostgreSQL's historical LZ compression) for compressing full-page images. The `XLogCompressBackupBlock` function in [`xloginsert/src/lib.rs`](https://github.com/malisper/pgrust/blob/main/xloginsert/src/lib.rs) handles this compression when the "hole" removed from a standard page still leaves a large enough image to benefit from compression, though the specific compression routine may be extended to support newer algorithms in future versions.