What Is the Memory Context System (mcx) in pgrust? Hierarchical Memory Management Explained

The memory context system (mcx) in pgrust is a Rust implementation of PostgreSQL's hierarchical arena allocator that groups allocations into tree-structured contexts, tracks memory usage with accounting counters, and provides deterministic cleanup through reset callbacks while leveraging Rust's borrow checker to prevent use-after-free errors.

pgrust uses the memory context system (mcx) to bring PostgreSQL's battle-tested memory management patterns into safe Rust. As implemented in the crates/_support/mcx crate, mcx provides a hierarchical arena allocator that replaces raw malloc/free calls with explicit, scoped memory pools called contexts. Understanding how mcx manages allocation lifecycles, accounting, and safety boundaries is essential for working with the pgrust codebase or extending it with new procedural language handlers.

Core Components of the mcx Memory Context System

The mcx crate centers on two primary types defined in crates/_support/mcx/src/lib.rs.

MemoryContext is the heavyweight arena object that owns a specific backend allocator (such as malloc or a bump allocator) and maintains accounting data. It tracks self_used bytes for its own allocations and subtree_used for aggregate usage across all descendants. When a context drops, it fires reset callbacks and propagates residual byte counts up the ancestor chain to keep parent counters consistent. The struct definition lives at lines 370-380.

Mcx<'mcx> is a lightweight zero-cost handle represented as &'mcx MemoryContext. It implements the standard Allocator trait, allowing collections like PgVec, PgBox, and PgHashMap to allocate directly inside the context without cloning the full struct. This handle appears at lines 918-923.

Context Creation Methods

You initialize a hierarchy through specific constructors:

• MemoryContext::new(name) creates a fresh top-level context backed by the system allocator.
• MemoryContext::new_bump(name) creates a bump arena optimized for fast allocation and bulk reset.

Both methods return an owned MemoryContext that acts as the root of a new allocation tree.

How the Memory Context System (mcx) Allocates Memory

When code requests memory through Mcx, the system performs accounting before delegation. The process follows these steps:

1. The collection calls new_in(ctx.mcx()), invoking the Mcx Allocator implementation.
2. The context charges the requested byte size to its internal counters (self_used, subtree_used).
3. If sufficient headroom exists, the request forwards to the stored backend (Backend::Malloc or Backend::Bump).
4. If allocation fails, the charge is undone and Mcx::oom raises a PgError out-of-memory condition.

Because Mcx holds a reference to its owning MemoryContext, Rust's borrow checker enforces two critical safety properties: allocations cannot outlive their context, and the context cannot be reset while any live allocation borrows it. These guarantees are explicitly documented in the crate at lines 998-1018.

Hierarchy, Accounting, and Cleanup in mcx

Memory contexts form a tree structure where each node tracks its parent through ancestors() and aggregates descendant usage in subtree_used. This design enables O(1) memory accounting for entire subtrees without scanning individual allocations.

Reset callbacks mirror PostgreSQL's MemoryContextDelete behavior. When a context drops—or when reset() or delete() is called explicitly—the system invokes registered callbacks, frees all owned allocations, and propagates any residual byte counts upward. This propagation ensures ancestor counters remain accurate even when child arenas vanish. The Drop implementation handling this cleanup appears at lines 10-34 in crates/_support/mcx/src/lib.rs.

MemoryContext::reset() clears all allocations while keeping the arena alive for reuse, ideal for per-tuple scratch spaces. **MemoryContext::delete()` fully destroys the arena and recursively deletes its children.

Practical Usage Examples

The following patterns demonstrate typical mcx workflows in pgrust.

Create a short-lived context for isolated operations:

// Create a short-lived context for a single operation.
let ctx = mcx::MemoryContext::new("short-lived");

// Allocate a vector that lives inside the context.
let mut v = mcx::PgVec::<u8>::new_in(ctx.mcx());

// Push some data – the allocation is charged to `ctx`.
v.push(42);

// The context can be reset only after `v` is dropped.
drop(v);
ctx.reset();          // clears all allocations at once

Use a bump arena for high-frequency temporary data:

// A per-tuple bump arena (fast allocation, bulk reset).
let per_tuple = mcx::MemoryContext::new_bump("per-tuple");

// Allocate a temporary string inside the bump arena.
let tmp = mcx::PgString::from_in(b"hello", per_tuple.mcx());

// Use the string while the arena lives…
println!("{}", tmp);

// Reset the arena for the next tuple; all allocations are reclaimed.
per_tuple.reset();

Cross-Crate Integration

The mcx system pervades pgrust's procedural language and SPI layers. Key integration points include:

• crates/_support/mcx/src/lib.rs – The core implementation containing MemoryContext, Mcx, and the Drop logic.
• crates/pl/plpgsql/src/plpgsql_scanner/src/lib.rs – The PlpgsqlScanner holds an Mcx handle to allocate lexer tokens directly inside the compilation context.
• crates/pl/plpgsql/src/handler/src/lib.rs – Creates scratch MemoryContext instances for PL/pgSQL compilation and validation phases.
• crates/pl/plpgsql/src/funcs/src/lib.rs – Passes Mcx references through the SPI layer, demonstrating how memory contexts cross crate boundaries while maintaining safety.

Summary

• MemoryContext owns a backend allocator and tracks usage via self_used and subtree_used counters in a hierarchical tree.
• Mcx<'mcx> acts as a lightweight Allocator trait handle that binds collection lifecycles to their context through Rust references.
• The system charges bytes before allocating and rollbacks on failure, converting OOM conditions into PgError.
• Rust's borrow checker prevents use-after-reset and use-after-free by tying allocation lifetimes to context handles.
• Reset callbacks and hierarchical accounting allow bulk cleanup while updating ancestor counters, mirroring PostgreSQL's native MemoryContext behavior.

Frequently Asked Questions

What is the difference between MemoryContext::reset() and MemoryContext::delete()?

reset() clears all allocations within the context but preserves the arena structure, allowing immediate reuse without reallocation overhead. delete() fully destroys the arena, invokes all reset callbacks, and recursively removes the context from its parent's child list, permanently reclaiming the backing memory.

Why does pgrust use Mcx handles instead of passing MemoryContext directly?

Mcx is a zero-cost reference (&MemoryContext) that implements Rust's standard Allocator trait. This design lets collections allocate directly through the context while the borrow checker ensures the context outlives every allocation made through it. Passing MemoryContext by value would transfer ownership and prevent multiple collections from sharing the same arena.

How does mcx prevent memory leaks when allocation fails?

Before delegating to the backend (malloc or bump), the context charges the requested size to its accounting counters. If the backend returns null or fails, the charge is undone within the same scope, ensuring counters reflect actual usage. Failed allocations trigger Mcx::oom, which produces a PgError without leaking the accounting reservation.

Can child contexts exceed their parent's memory limits?

While the system tracks usage hierarchically via subtree_used, current enforcement occurs at the individual context level during the charge phase. A child context exceeding its own limit triggers OOM, but pgrust's design allows ancestors to query aggregate usage through subtree_used to implement policy-level limits higher in the tree.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}