SharpEmu Memory Architecture and Guest-to-Host Address Translation Explained

SharpEmu implements a flat 64-bit guest address space on top of the host's virtual memory, using the PhysicalVirtualMemory class to translate guest addresses to host virtual addresses via a sorted region table that maps guest ranges directly to host memory allocations.

SharpEmu is a high-performance emulator that virtualizes a 64-bit guest environment on top of host operating system resources. Understanding SharpEmu memory architecture requires examining how the PhysicalVirtualMemory class bridges the gap between guest address expectations and host physical resources. This article breaks down the guest-to-host address translation mechanism, allocation strategies, and synchronization primitives implemented in the source code.

The PhysicalVirtualMemory Core

The PhysicalVirtualMemory class in src/SharpEmu.Core/Memory/PhysicalVirtualMemory.cs serves as the central memory manager, implementing three critical interfaces: IVirtualMemory for generic operations, IGuestMemoryAllocator for guest-side allocations, and IGuestAddressSpace for the translation layer itself. This class maintains a private List<MemoryRegion> called _regions that tracks every allocated, reserved, or mapped segment in the guest address space.

Each MemoryRegion stores the VirtualAddress (which represents the host virtual address backing the guest range), size, protection flags, and execution permissions. The class delegates actual host system calls to an IHostMemory instance, which abstracts platform-specific implementations like HostMemory.Alloc, HostMemory.Free, and HostMemory.Protect.

How Guest-to-Host Address Translation Works

Region Bookkeeping and MemoryRegion Structure

Every memory operation creates or updates a MemoryRegion entry containing the host virtual address, size, protection level, and reservation status. The _regions list is maintained in sorted order to enable fast lookup operations during address resolution. When the emulator needs to translate a guest address, it searches this list to find the containing region, then calculates the offset from the region's base to determine the exact host pointer.

The Translation Process via TryResolveAddressSpace

Translation occurs through the TryResolveAddressSpace method, which internally calls KernelVirtualRangeAllocator.TryResolveAddressSpace as defined in src/SharpEmu.Libs/Kernel/KernelVirtualRangeAllocator.cs. The algorithm walks the sorted _regions list to locate which MemoryRegion contains the queried guest address. Once found, the method returns the IGuestAddressSpace instance owning that region, allowing direct access to the backing host memory.

Memory Allocation Strategy

Page-Level Allocation and Alignment

All allocations align to a 4 KB page boundary (PageSize = 0x1000). The TryAllocateAtExact method (lines 154-190 in PhysicalVirtualMemory.cs) handles requests for specific guest addresses by first validating the range is available, then calling the host memory allocator to reserve or commit physical pages. The returned host pointer is stored in a new MemoryRegion where VirtualAddress equals the host address, establishing the guest-to-host mapping.

Large Data Handling

When allocation requests exceed LargeDataReserveThreshold (1 GiB), the allocator switches to a lazy reservation strategy. It reserves a large contiguous region in the host address space but commits individual pages only when accessed. This approach minimizes host-side fragmentation while maintaining a contiguous guest address space appearance. The mechanism uses LazyReservePrimeBytes and LazyReservePrimeChunkBytes parameters to control commit granularity.

Guest Allocation Arena Isolation

To prevent conflicts with guest-loaded images, SharpEmu reserves a dedicated arena at GuestAllocationArenaAddress = 0x00006000_0000_0000 for internal emulator allocations. This ensures that runtime allocations by the emulator itself never collide with the guest's view of memory, maintaining strict separation between emulator infrastructure and emulated program data.

Thread Safety and Synchronization

Concurrent access to the memory subsystem is protected by a ReaderWriterLockSlim instance named _gate, which guards the _regions list during read and write operations. Additionally, separate lock objects protect allocation search hints and the arena's free-range map. This locking strategy allows multiple threads to resolve addresses or read memory concurrently while ensuring exclusive access during allocation or deallocation operations.

Code Examples

// Allocate 4 KB of executable guest memory at a specific address
ulong guestAddr;
bool ok = vm.TryAllocateAtExact(0x1400_0000_0000, 0x1000, executable: true, out guestAddr);
// guestAddr == 0x1400_0000_0000 if allocation succeeded

// Allocate a writable data buffer of 256 KB (address chosen by the allocator)
ulong dataAddr = vm.Allocate(0x40000, executable: false);

// Resolve a guest address to its backing host region for debugging
if (PhysicalVirtualMemory.TryResolveAddressSpace(vm, out var guestSpace))
{
    // guestSpace implements IGuestAddressSpace and provides region metadata
}

Summary

• SharpEmu uses a flat 64-bit guest address space managed by the PhysicalVirtualMemory class in src/SharpEmu.Core/Memory/PhysicalVirtualMemory.cs.
• Guest-to-host translation relies on a sorted List<MemoryRegion> where each region's VirtualAddress field stores the actual host virtual address backing the guest range.
• The TryResolveAddressSpace method walks region tables to map guest pointers to host memory, enabling on-the-fly address translation.
• Allocations align to 4 KB pages, with special handling for large data (>1 GB) using lazy commit strategies to reduce fragmentation.
• Thread safety is enforced via ReaderWriterLockSlim (_gate) protecting the region list, with separate locks for allocation hints.

Frequently Asked Questions

How does SharpEmu prevent guest addresses from colliding with host memory?

SharpEmu isolates guest allocations by using a dedicated GuestAllocationArenaAddress at 0x00006000_0000_0000, ensuring all emulator internal allocations occur in a reserved high-memory region separate from typical guest-loaded images.

What happens when a guest address is not found in the region list?

If TryResolveAddressSpace fails to locate a MemoryRegion containing the queried address, the translation fails and returns false, indicating the guest address is unmapped or invalid in the current address space context.

Why does SharpEmu use lazy commitment for large allocations?

When allocations exceed the 1 GB LargeDataReserveThreshold, the system reserves virtual address space upfront but commits physical pages lazily using LazyReservePrimeBytes parameters, minimizing host memory fragmentation while presenting a contiguous guest address space.

Which interface defines the contract for host memory operations?

The IHostMemory interface in src/SharpEmu.HLE/Host/IHostMemory.cs abstracts platform-specific memory functions, with concrete implementations like HostMemory handling Windows or POSIX syscalls for allocation, protection, and deallocation.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}