How LiteBox Handles Memory Management and Page Allocation: A Deep Dive

LiteBox implements a generic virtual-memory manager through the PageManager type, which coordinates platform-specific page allocation via the PageManagementProvider trait to handle Linux-compatible syscalls like mmap, mprotect, and brk.

LiteBox is a lightweight, cross-platform sandboxing runtime that requires precise control over virtual memory to safely execute untrusted code. Understanding how LiteBox handles memory management and page allocation reveals how it abstracts platform differences while providing a consistent POSIX-like interface. The architecture centers on a generic PageManager that orchestrates page tables, permission bits, and physical backing stores through pluggable platform providers.

Architecture Overview: PageManager and Platform Abstraction

At the heart of LiteBox's memory subsystem is the PageManager<Platform, ALIGN> struct defined in litebox/src/mm/mod.rs. This generic type holds a read-write lock around a Vmem object that tracks all virtual-memory ranges and their attributes.

The PageManager delegates actual physical page operations to the PageManagementProvider trait, which is implemented in platform-specific crates like litebox_platform_multiplex. This separation allows LiteBox to run on Linux userland, Windows, hypervisors (LVBS), or trusted execution environments (OP-TEE) without changing the core memory management logic.

The PageManager Core Implementation

The PageManager provides a high-level façade for creating, modifying, and destroying virtual memory mappings. All operations are marked unsafe because they manipulate raw pointers; the caller (typically the syscall layer) guarantees that the target memory is not concurrently accessed.

Creating and Removing Mappings

To allocate new virtual memory, PageManager offers specialized methods that install specific MemoryRegionPermissions:

• create_writable_pages – Allocates readable and writable anonymous memory.
• create_executable_pages – Allocates memory marked for code execution.
• create_readable_pages – Allocates read-only data pages.
• create_inaccessible_pages – Allocates pages with no permissions (guard pages).

These methods call the underlying Vmem::create_pages, run a user-provided initialization closure, then optionally finalize permissions. The implementation resides in litebox/src/mm/mod.rs between lines 24 and 78.

To release memory, remove_pages validates alignment, constructs a PageRange, and forwards to Vmem::remove_mapping (lines 48-57).

Modifying Permissions and Remapping

The change_page_permissions method (lines 90-102) wraps Vmem::protect_mapping to apply new permission bits to an existing range.

For resizing or moving mappings, remap_pages (lines 104-146) implements the logic for mremap. It either expands the current range in-place or allocates a new region and copies data when may_move is allowed.

Program Break and Memory Reset

The brk method (lines 162-194) adjusts the end of the data segment by allocating or releasing pages, implementing the traditional Unix program break.

For memory reset operations like MADV_DONTNEED or MADV_FREE, reset_pages (lines 124-133) calls Vmem::reset_pages without unmapping, optionally restricting the operation to anonymous pages.

Linux Syscall Interface

LiteBox exposes these internal primitives through a Linux-compatible syscall layer, allowing unmodified Linux binaries to run inside the sandbox.

Anonymous and File-Backed Mappings

The do_mmap function in litebox_common_linux/src/mm.rs (lines 29-84) translates Linux MAP_* flags into CreatePagesFlags. It validates the optional hint address, selects the appropriate create_*_pages method based on prot bits, and forwards the user-provided initializer closure.

Syscall entry points in litebox_shim_linux/src/syscalls/mm.rs (lines 15-57) parse arguments and delegate to these common helpers:

pub(crate) fn sys_mmap(&self, addr: usize, len: usize,
    prot: ProtFlags, flags: MapFlags,
    fd: i32, offset: usize) -> Result<MutPtr<u8>, Errno> { … }

Permission Changes and Resizing

Thin wrappers in litebox_common_linux/src/mm.rs (lines 86-106) validate alignment and forward to PageManager methods:

• sys_munmap – Calls PageManager::remove_pages.
• sys_mprotect – Calls PageManager::change_page_permissions.
• sys_mremap – Calls PageManager::remap_pages.
• sys_brk – Calls PageManager::brk.
• sys_madvise – Calls PageManager::reset_pages.

Physical-to-Virtual Contiguous Mapping

For device drivers or kernel code requiring vmap() semantics, LiteBox defines the VmapManager trait in litebox_common_linux/src/vmap.rs (lines 14-34).

This trait provides:

unsafe fn vmap(&self, pages: &PhysPageAddrArray<ALIGN>, 
    perms: PhysPageMapPermissions) -> Result<PhysPageMapInfo<ALIGN>, PhysPointerError>;
unsafe fn vunmap(&self, info: PhysPageMapInfo<ALIGN>) -> Result<(), PhysPointerError>;

Platform-specific implementations (e.g., in litebox_platform_multiplex) map an array of physical pages into a virtually contiguous region. The default trait methods return UnsupportedOperation for platforms lacking this capability.

Practical Code Examples

Anonymous Memory Mapping (mmap)

The following example demonstrates allocating anonymous writable memory through the syscall interface:

let task = init_platform(None);                     // create a test Task
let addr = task
    .sys_mmap(
        0,                                         // let the kernel pick the address
        0x2000,                                    // 8 KiB
        ProtFlags::PROT_READ | ProtFlags::PROT_WRITE,
        MapFlags::MAP_ANON | MapFlags::MAP_PRIVATE,
        -1,                                        // fd ignored for anonymous
        0,
    )
    .unwrap();                                    // => MutPtr<u8>

addr.write_slice_at_offset(0, &[0xAA; 0x2000]).unwrap();   // write to the mapping
assert_eq!(addr.read_at_offset(0x1000).unwrap(), 0xAA);
task.sys_munmap(addr, 0x2000).unwrap();          // clean up

Underlying path: sys_mmap → do_mmap_anonymous → do_mmap → PageManager::create_writable_pages.

Changing Page Permissions (mprotect)

To modify permissions on an existing mapping:

// Assume `addr` points to a previously-mapped region.
task.sys_mprotect(addr, 0x2000, ProtFlags::PROT_READ).unwrap();
// The pages are now read-only; a subsequent write will fault.

This invokes PageManager::make_pages_readable via the change_page_permissions helper.

Resizing Mappings (mremap)

Expanding or moving a mapping with mremap:

let new_addr = task
    .sys_mremap(
        addr,                // old base
        0x2000,              // old size
        0x4000,              // new size
        MRemapFlags::MREMAP_MAYMOVE,
        0,                   // ignored for non-FIXED
    )
    .unwrap();

PageManager::remap_pages handles the logic, either extending in-place or relocating when may_move is permitted.

Contiguous Physical Mapping (vmap)

For platform-specific contiguous physical-to-virtual mapping:

// Platform-specific impl provides `vmap`.  Here we only show the trait usage.
let phys_pages: [PhysPageAddr<4096>; 2] = [/* … */];
let vmap_info = platform
    .vmap(&phys_pages, PhysPageMapPermissions::READ | PhysPageMapPermissions::WRITE)
    .expect("vmap unsupported on this platform");

// `vmap_info.base` is a contiguous virtual address region that can be used like a slice.
let slice = unsafe { core::slice::from_raw_parts_mut(vmap_info.base, vmap_info.size) };
slice[0] = 0x55;

The actual implementation resides in platform crates such as litebox_platform_multiplex.

Summary

LiteBox handles memory management and page allocation through a layered architecture that separates policy from mechanism:

• Generic Core: The PageManager in litebox/src/mm/mod.rs provides a thread-safe, generic interface for creating, modifying, and destroying virtual memory mappings.
• Platform Abstraction: The PageManagementProvider trait enables porting to Linux, Windows, hypervisors, and TEEs without modifying core logic.
• Linux Compatibility: The litebox_common_linux and litebox_shim_linux crates translate POSIX syscalls (mmap, mprotect, mremap, brk) into internal PageManager operations.
• Advanced Mapping: The VmapManager trait supports contiguous physical-to-virtual mappings for device drivers and kernel modules.

This design allows LiteBox to sandbox untrusted code with precise memory control while maintaining compatibility with existing Linux binaries.

Frequently Asked Questions

How does LiteBox differ from the Linux kernel's memory management?

LiteBox implements a userspace virtual memory manager that sits on top of the host's page allocation primitives rather than managing physical page tables directly. While the Linux kernel handles page faults, TLB shoots, and physical memory fragmentation, LiteBox's PageManager focuses on permission enforcement, address space layout, and syscall compatibility. This allows LiteBox to run as a sandbox on various platforms—including Windows and hypervisors—while presenting a consistent Linux-like interface to guest code.

What is the role of the PageManagementProvider trait?

The PageManagementProvider trait defines the low-level interface between LiteBox's generic memory manager and the underlying platform. It specifies methods like alloc_page, free_page, map_page, and unmap_page that the PageManager invokes to materialize virtual memory operations. Platform-specific crates (such as litebox_platform_multiplex for Linux userland or litebox_platform_optee for trusted execution environments) implement this trait, enabling the same high-level memory code to run across disparate hardware and OS targets.

How does LiteBox handle the mremap syscall without a traditional kernel?

LiteBox implements mremap through the PageManager::remap_pages method in litebox/src/mm/mod.rs (lines 104-146). When a process requests to resize a mapping—optionally allowing the kernel to move it via MREMAP_MAYMOVE—the PageManager either extends the existing virtual range if adjacent pages are free, or allocates a new region and copies the data. This logic mirrors the Linux kernel's mremap behavior but operates entirely within LiteBox's virtualized address space tracking, without requiring privileged page table modifications.

Can LiteBox map physically contiguous memory for device drivers?

Yes, through the VmapManager trait defined in litebox_common_linux/src/vmap.rs (lines 14-34). This interface provides vmap and vunmap methods that accept an array of physical page addresses and map them into a virtually contiguous region. Platform implementations (such as those in litebox_platform_multiplex) handle the specific mechanism for establishing this mapping, which is essential for device drivers that require contiguous buffers for DMA operations. If the underlying platform cannot support this operation, the trait methods return UnsupportedOperation.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}