# Rust Implementation of the Buffer Manager and Storage Layer in pgrust

> Explore the Rust implementation of PostgreSQL's buffer manager and storage layer in pgrust. Learn how the BufferManager struct centralizes operations for efficient data management.

- Repository: [Michael Malis/pgrust](https://github.com/malisper/pgrust)
- Tags: internals
- Published: 2026-07-13

---

**The pgrust project reimplements PostgreSQL's shared-buffer manager and storage subsystem in pure Rust using a seam-based architecture where the `BufferManager` struct in [`crates/backend/storage/buffer/bufmgr/src/mgr.rs`](https://github.com/malisper/pgrust/blob/main/crates/backend/storage/buffer/bufmgr/src/mgr.rs) centralizes descriptor arrays, page storage, and content locks, exposing operations via testable seam functions registered at runtime.**

The **pgrust** repository provides a Rust-native implementation of PostgreSQL's buffer management and storage layer, translating the original C architecture from [`storage/buffer/bufmgr.c`](https://github.com/malisper/pgrust/blob/main/storage/buffer/bufmgr.c) into a modular, testable design. This implementation preserves PostgreSQL's behavioral semantics while leveraging Rust's type safety and memory guarantees. The core system coordinates buffer descriptors, page memory, and per-backend pinning through a seam-based API that decouples interface from implementation.

## Core Architecture of the Buffer Manager

### The BufferManager Singleton

In [`crates/backend/storage/buffer/bufmgr/src/mgr.rs`](https://github.com/malisper/pgrust/blob/main/crates/backend/storage/buffer/bufmgr/src/mgr.rs), the **BufferManager** struct owns the global state backing the shared buffer pool. It maintains the **descriptor array** (`BufferDesc`) tracking each buffer's metadata, the page-byte array holding actual `BLCKSZ`-sized pages, per-buffer **content locks** (`lwlock::LWLock`), and the per-backend pin-count map (`refcount`). All public operations delegate to a singleton instance accessed via `BufferManager::global_expect()`, ensuring every call sees a consistent view of the pool.

### Seam Functions and Public API

The buffer manager exposes operations via **seam** functions defined in [`crates/backend/storage/buffer/bufmgr/src/lib.rs`](https://github.com/malisper/pgrust/blob/main/crates/backend/storage/buffer/bufmgr/src/lib.rs). These seams are registered at runtime through `init_seams()`, allowing higher-level crates to call buffer operations without depending on concrete implementation details. Key seams include:

- `read_buffer`: Synchronous `ReadBuffer(rel, blkno)` that pins a buffer and returns its `Buffer` id (lines 63-70)
- `release_buffer`: Drops one pin via `ReleaseBuffer(buf)` (lines 69-74)
- `lock_buffer` and `lock_buffer_exclusive`: Acquire content locks in shared or exclusive mode (lines 107-110)
- `mark_buffer_dirty`: Sets the dirty flag on a pinned buffer required for write-back (lines 55-62)
- `extend_buffered_rel`: Allocates a new block on a relation and returns a write-locked, pinned buffer (lines 66-75)
- `flush_one_buffer`: Writes a single dirty buffer to disk (lines 80-87)
- `init_buffer_manager_access`: Initializes the per-backend pin map and registers cleanup hooks (lines 85-90)

## Storage Layer Components

### Primitive Types and Abstractions

The storage layer relies on type definitions in [`crates/_support/types/storage/src/lib.rs`](https://github.com/malisper/pgrust/blob/main/crates/_support/types/storage/src/lib.rs), which provides **primitive types** like `RelFileLocator`, `BlockNumber`, `ForkNumber`, and `ReadBufferMode`. These types map PostgreSQL relations onto physical on-disk file identifiers and are used throughout the storage code to maintain type safety across the boundary between logical tables and physical storage.

### Page Management and Storage Manager

Page-level utilities reside in [`crates/backend/storage/page/src/lib.rs`](https://github.com/malisper/pgrust/blob/main/crates/backend/storage/page/src/lib.rs), implementing functions such as `page_init`, `page_set_lsn`, and `page_is_new` that operate on raw `&mut [u8]` slices. The **storage manager** (`smgr`) in [`crates/backend/storage/smgr/src/lib.rs`](https://github.com/malisper/pgrust/blob/main/crates/backend/storage/smgr/src/lib.rs) handles whole-page I/O to the underlying file system, exposing `smgrextend`, `smgrread`, and `smgrwrite`. The buffer manager invokes these functions on cache misses to read pages from disk or write them back during flushing.

### Shared Memory Infrastructure

Shared-memory allocation for the buffer pool is managed in [`crates/backend/storage/ipc/src/ipci_core.rs`](https://github.com/malisper/pgrust/blob/main/crates/backend/storage/ipc/src/ipci_core.rs). This module calculates `buffer_manager_shmem_size` and performs `buffer_manager_shmem_init` to establish the shared memory segment that backs the global buffer pool, enabling multi-process access to the same physical page cache.

## Operational Workflow

The buffer manager and storage layer interact through a four-stage pipeline during backend operation.

1. **Backend Startup**: The system calls `init_buffer_manager_access` from [`crates/backend/utils/init/postinit/src/lib.rs`](https://github.com/malisper/pgrust/blob/main/crates/backend/utils/init/postinit/src/lib.rs) to allocate the per-process pin map and register the `UnlockBuffers` cleanup hook.

2. **Relation Access**: When executors or extensions require a block, they invoke the `read_buffer` seam. If the block resides in the pool, the buffer is pinned immediately; otherwise, the manager calls `smgrread` to fetch the page from disk, allocates a descriptor, and returns the buffer id.

3. **Modification**: After changing page content, callers invoke `mark_buffer_dirty` to set the buffer's dirty flag. During transaction commit or checkpoint, `flush_one_buffer` (or the background writer) writes modified pages back to stable storage via `smgrwrite`.

4. **Extension**: The `extend_buffered_rel` seam creates new blocks on a relation's fork, locks the new page exclusively, and returns a buffer ready for data insertion, mirroring PostgreSQL's `ExtendBufferedRel` behavior.

All operations remain stateless from the caller's perspective, with the global `BufferManager` holding mutable state while seams provide a clean, testable API surface.

## Working with the Buffer Manager in Practice

The seam-based design allows code to interact with the buffer pool through simple function calls that resolve to the global `BufferManager` instance. The `with_buffer_page` function provides safe access to underlying page bytes, mirroring the C macro `BufferGetPage`.

```rust
use pgrust::bufmgr_seams as bufmgr;
use pgrust::types_storage::storage::{ReadBufferMode, RelFileLocator};
use pgrust::types_storage::buf::BufferAccessStrategy;
use pgrust::types_core::primitive::{BlockNumber, ForkNumber};

/// Read a page from a relation (pinning it)
fn read_page_example(rel: &rel::Relation, blkno: BlockNumber) -> PgResult<Buffer> {
    // Calls the `read_buffer` seam → `ReadBuffer(rel, blkno)`
    bufmgr::read_buffer::call(rel, blkno)
}

/// Extend a relation by one block (useful for INSERT)
fn extend_one_block(rel: &rel::Relation, fork: ForkNumber) -> PgResult<Buffer> {
    // Mirrors `ExtendBufferedRel(rel, fork, …)`
    bufmgr::extend_buffered_rel::call(rel, fork)
}

/// Mark the buffer dirty and write it back explicitly
fn dirty_and_flush(buf: Buffer) -> PgResult<()> {
    bufmgr::mark_buffer_dirty::call(buf);
    // Flush the single dirty buffer to disk
    bufmgr::flush_one_buffer::call(buf)
}

/// Work with the underlying page bytes safely
fn modify_page(buf: Buffer) -> PgResult<()> {
    // `with_buffer_page` gives a mutable slice of the page content.
    bufmgr::with_buffer_page::call(buf, &mut |page| {
        // Example: zero-out the whole page
        for byte in page.iter_mut() {
            *byte = 0;
        }
        Ok(())
    })
}

```

These calls work uniformly whether the backing storage is an in-memory simulation or a live PostgreSQL data directory, as the seams delegate to `BufferManager::global_expect()`.

## Summary

- The **pgrust** buffer manager reimplements PostgreSQL's [`bufmgr.c`](https://github.com/malisper/pgrust/blob/main/bufmgr.c) in Rust, maintaining architectural compatibility while improving testability through seams.
- The **`BufferManager`** struct in [`crates/backend/storage/buffer/bufmgr/src/mgr.rs`](https://github.com/malisper/pgrust/blob/main/crates/backend/storage/buffer/bufmgr/src/mgr.rs) centralizes buffer descriptors, page storage, content locks, and per-backend pinning maps.
- **Seam functions** in [`crates/backend/storage/buffer/bufmgr/src/lib.rs`](https://github.com/malisper/pgrust/blob/main/crates/backend/storage/buffer/bufmgr/src/lib.rs) provide a runtime-wired API that decouples callers from implementation details.
- The **storage layer** comprises primitive types (`_support/types/storage`), page utilities (`storage/page`), the storage manager (`storage/smgr`), and shared-memory infrastructure (`storage/ipc`).
- Operations follow the PostgreSQL workflow: **initialize** via `init_buffer_manager_access`, **access** via `read_buffer`, **modify** via `mark_buffer_dirty`, and **extend** via `extend_buffered_rel`.

## Frequently Asked Questions

### What is a seam in pgrust's buffer manager?

A **seam** is a small, testable function exported from [`crates/backend/storage/buffer/bufmgr/src/lib.rs`](https://github.com/malisper/pgrust/blob/main/crates/backend/storage/buffer/bufmgr/src/lib.rs) that wraps buffer manager operations. Seams are registered at runtime via `init_seams()`, allowing higher-level crates to call functions like `read_buffer` or `mark_buffer_dirty` without importing implementation details from [`mgr.rs`](https://github.com/malisper/pgrust/blob/main/mgr.rs). This pattern enables dependency injection for unit testing while maintaining PostgreSQL-compatible semantics.

### How does pgrust handle buffer pinning and unpinning?

Buffer pinning is managed through the **per-backend pin-count map** (`refcount`) stored in the global `BufferManager`. When code calls the `read_buffer` seam, the manager increments the pin count for that buffer descriptor. Calling `release_buffer` decrements the count. The `UnlockBuffers` cleanup hook registered during `init_buffer_manager_access` ensures pins are released automatically if a backend exits unexpectedly, preventing resource leaks.

### What is the relationship between the buffer manager and smgr?

The **storage manager** (`smgr`) in [`crates/backend/storage/smgr/src/lib.rs`](https://github.com/malisper/pgrust/blob/main/crates/backend/storage/smgr/src/lib.rs) operates below the buffer manager, handling direct file I/O. When the buffer manager encounters a cache miss during `read_buffer`, it invokes `smgrread` to fetch the page from disk. Similarly, `flush_one_buffer` delegates to `smgrwrite` to persist dirty pages. This separation allows the buffer manager to focus on caching and concurrency while `smgr` manages physical storage layout.

### How does pgrust ensure thread safety in the buffer pool?

Thread safety is achieved through **content locks** (`lwlock::LWLock`) associated with each buffer descriptor and the global singleton pattern of `BufferManager::global_expect()`. The seam-based API ensures that all mutable state access passes through the centralized manager, while Rust's ownership model prevents data races on the page slices handed out via `with_buffer_page`. Shared memory initialization in [`ipci_core.rs`](https://github.com/malisper/pgrust/blob/main/ipci_core.rs) establishes the proper memory layout for multi-process access as required by PostgreSQL's architecture.