# How CubeSandbox Uses CubeHypervisor and KVM MicroVMs for Hardware-Level Isolation > Discover how CubeSandbox utilizes CubeHypervisor and KVM MicroVMs for robust hardware-level isolation. Learn how it protects your system from untrusted workloads. - Repository: [Tencent Cloud/CubeSandbox](https://github.com/TencentCloud/CubeSandbox) - Tags: internals - Published: 2026-07-16 --- **CubeSandbox leverages CubeHypervisor, a lightweight KVM-based hypervisor, to run minimal MicroVMs that utilize CPU virtualization extensions (VT-x/AMD-V) to enforce strict hardware-level isolation between untrusted workloads and the host system.** CubeSandbox, developed by TencentCloud, achieves secure workload isolation through tight integration with CubeHypervisor, which is included as a Git submodule referenced in the repository’s `.gitmodules` file. This specialized hypervisor creates tiny MicroVMs—typically consuming only ~10 MiB of memory—that harness Linux KVM and hardware virtualization extensions to provide near-native performance while maintaining robust security boundaries between sandboxes. ## Architecture Overview The isolation architecture relies on a thin user-space layer interacting directly with the host kernel’s virtualization capabilities. - **Host Kernel Layer**: The Linux kernel provides the KVM driver, exposing `/dev/kvm` to user space. - **CubeHypervisor Layer**: This lightweight component opens `/dev/kvm`, creates VMs using `KVM_CREATE_VM`, and configures minimal virtual machines with vCPUs, memory, and virtio devices. - **MicroVM Layer**: Each sandbox runs as a stripped-down Linux kernel with a tiny user-space runtime, executing workloads in isolated virtual address spaces protected by **VT-x/AMD-V** hardware extensions. This design ensures that workloads cannot access the host’s memory or other VMs’ memory spaces, providing true **hardware-level isolation** with minimal overhead. ## MicroVM Lifecycle Management CubeHypervisor manages the complete lifecycle of sandboxed environments through precise KVM ioctl operations. ### VM Creation The hypervisor initializes by opening `/dev/kvm` and creating a new VM instance via the `KVM_CREATE_VM` ioctl. In [`hypervisor.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/hypervisor.go), the core logic sets up the VM’s memory layout and virtual CPU configuration, allocating resources while maintaining a minimal footprint. ### Boot and Execution A minimal kernel image is loaded into the VM’s memory space. The VM boots rapidly due to the stripped-down kernel and tiny initramfs. Execution begins with the `KVM_RUN` ioctl, which enters the guest context and runs the workload until completion or interruption. ### Snapshot and Restore **Snapshot Creation**: During operation, CubeHypervisor captures the VM’s memory state using `KVM_GET_MEMORY_REGION` and related ioctls. The [`snapshot.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/snapshot.go) file implements incremental snapshotting that tracks only dirty pages, writing changes to persistent storage efficiently. **Restore Process**: To resume a sandbox, the snapshot file is memory-mapped back into the guest physical address space using `mmap`, and the VM resumes execution via `KVM_RUN`. This allows for fast checkpoint/restore operations during debugging or migration. ### Resource Teardown When a sandbox completes, the hypervisor halts the VM and releases all associated resources, including memory mappings and file descriptors, ensuring no resource leakage between sandbox instances. ## Key Implementation Files The hypervisor’s functionality is organized into specific source files that handle distinct responsibilities: - **[`hypervisor.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/hypervisor.go)**: Contains the core logic for initializing the hypervisor, opening `/dev/kvm`, and managing the VM creation and configuration workflows. - **[`vm.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/vm.go)**: Implements the VM abstraction layer, including vCPU setup, memory region registration, and virtio device configuration. - **[`snapshot.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/snapshot.go)**: Handles memory snapshot operations and incremental diff tracking, enabling efficient state capture and restoration. - **[`api.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/api.go)**: Exposes the public Go API consumed by CubeSandbox’s higher-level components, providing methods for lifecycle management and resource accounting. ## Integration with CubeSandbox's Cubelet CubeSandbox’s **Cubelet** component interacts with CubeHypervisor through the Go API defined in [`api.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/api.go). This integration allows Cubelet to: 1. **Launch** sandboxes as MicroVMs with specified resource constraints. 2. **Pause** active sandboxes by triggering snapshot creation followed by VM shutdown. 3. **Resume** sandboxes by restoring snapshots and restarting VMs via `KVM_RUN`. 4. **Manage** incremental snapshots for fast state transitions during development workflows. This tight coupling enables CubeSandbox to deliver a container-like developer experience while maintaining hardware-level isolation guarantees. ## Security Guarantees CubeHypervisor provides robust security through architectural constraints: - **Hardware Isolation**: Each MicroVM operates in its own virtual address space enforced by CPU virtualization extensions, preventing memory leaks or unauthorized access between VMs and the host. - **Minimal Attack Surface**: The hypervisor exposes only essential KVM ioctls required for VM lifecycle management, reducing the codebase vulnerable to exploitation. - **Snapshot Integrity**: Snapshots are written atomically and can be cryptographically signed by CubeSandbox components to verify they have not been tampered with during storage or transmission. ## Summary - CubeSandbox uses **CubeHypervisor** (included as a Git submodule) to provide **hardware-level isolation** via KVM-based MicroVMs. - MicroVMs run with minimal footprints (~10 MiB) using **VT-x/AMD-V** extensions and the Linux KVM driver (`/dev/kvm`). - The hypervisor manages the full lifecycle through [`hypervisor.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/hypervisor.go), [`vm.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/vm.go), and [`snapshot.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/snapshot.go), supporting creation, boot, incremental snapshots, and teardown. - **Cubelet** integrates via the [`api.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/api.go) Go API to orchestrate sandbox operations while maintaining a minimal attack surface. - Snapshots use dirty-page tracking for efficient checkpoint/restore capabilities. ## Frequently Asked Questions ### What hardware features enable isolation in CubeHypervisor? CubeHypervisor relies on CPU virtualization extensions (**VT-x for Intel, AMD-V for AMD**) to create isolated execution environments. These extensions allow the hypervisor to run MicroVMs in separate virtual address spaces, ensuring that sandboxed workloads cannot access host memory or other VMs’ memory regions, providing true hardware-level isolation. ### How does CubeHypervisor minimize startup latency? By using **MicroVMs** with stripped-down Linux kernels and minimal initramfs images, CubeHypervisor reduces the boot sequence to essentials only. The lightweight VM configuration (typically ~10 MiB RAM) and direct KVM integration via `/dev/kvm` ioctls eliminate the overhead associated with traditional full-system virtualization, enabling near-instantaneous sandbox startup. ### What is the relationship between CubeSandbox and CubeHypervisor? CubeHypervisor is included as a Git submodule within the CubeSandbox repository (referenced in `.gitmodules`). CubeSandbox’s **Cubelet** component uses the public Go API exposed in [`api.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/api.go) to manage MicroVMs, while CubeHypervisor handles the low-level KVM operations. This separation allows CubeSandbox to focus on sandbox orchestration while CubeHypervisor provides the underlying virtualization layer. ### How are MicroVM snapshots implemented? Snapshots are implemented in [`snapshot.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/snapshot.go) using KVM’s memory region ioctls to capture VM state. The system supports **incremental snapshots** that track only dirty pages (modified memory) rather than full memory dumps, significantly reducing storage requirements and enabling fast checkpoint/restore operations for paused sandboxes.