# How CubeHypervisor Manages KVM MicroVMs for Hardware-Level Isolation

> Discover how CubeHypervisor uses KVM MicroVMs to achieve hardware-level isolation. Learn about efficient CPU virtualization for minimal resource overhead. Explore the TencentCloud CubeSandbox.

- Repository: [Tencent Cloud/CubeSandbox](https://github.com/TencentCloud/CubeSandbox)
- Tags: internals
- Published: 2026-07-11

---

**CubeHypervisor creates and orchestrates tiny MicroVMs using Linux KVM to provide hardware-level isolation through CPU virtualization extensions while maintaining minimal resource overhead.**

CubeHypervisor is a lightweight, KVM-based hypervisor developed by TencentCloud as part of the CubeSandbox project. It enables CubeSandbox to run untrusted workloads in isolated MicroVMs that leverage hardware virtualization extensions (VT-x/AMD-V) for security. The hypervisor exposes a minimal Go API that integrates with CubeSandbox's Cubelet component to manage the complete lifecycle of these virtual machines.

## Architecture and KVM Integration

CubeHypervisor operates as a thin user-space layer atop the Linux KVM driver. According to the TencentCloud/cube-hypervisor source code, the architecture centers on direct interaction with `/dev/kvm` to create virtual machines with minimal attack surface.

The hypervisor initializes by opening the KVM device, then creates VM instances using the `KVM_CREATE_VM` ioctl. Each MicroVM typically allocates approximately **10 MiB of memory** and runs a stripped-down Linux kernel with a tiny user-space runtime. This minimal footprint reduces startup time and resource consumption while maintaining strict isolation boundaries enforced by the CPU's hardware virtualization extensions.

## MicroVM Lifecycle Management

CubeHypervisor manages the complete lifecycle of MicroVMs through five distinct phases implemented across [`hypervisor.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/hypervisor.go), [`vm.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/vm.go), and [`snapshot.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/snapshot.go).

### VM Creation and Configuration

In [`hypervisor.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/hypervisor.go), the hypervisor exposes the `New()` function to initialize the KVM context and `NewVM()` to create individual virtual machines. The creation process involves:

- Opening `/dev/kvm` and establishing the KVM context
- Invoking `KVM_CREATE_VM` to instantiate the VM
- Configuring the memory layout using `KVM_SET_MEMORY_REGION`
- Setting up **virtio devices** for I/O virtualization

The `VMConfig` struct allows specification of memory size (typically 16 MiB or less) and kernel image paths, enabling rapid provisioning of isolated execution environments.

### Boot Process

The [`vm.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/vm.go) file handles the VM abstraction, including vCPU setup and device initialization. When `vm.Run()` is called, CubeHypervisor loads a minimal kernel image into the allocated memory and invokes `KVM_RUN` to start execution. The VM boots quickly because the kernel and initramfs are stripped of unnecessary drivers and modules, resulting in a tiny memory footprint that reduces the attack surface.

### Snapshot and Restore Operations

[`snapshot.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/snapshot.go) implements incremental memory snapshots using `KVM_GET_MEMORY_REGION` and dirty page tracking. During operation, the hypervisor can:

- Capture the VM's memory state by reading guest physical pages
- Track dirty pages incrementally to minimize snapshot size
- Write atomic snapshots to disk for checkpoint/restore functionality

To resume, the snapshot file is `mmap`-ed back into the guest physical address space, and the VM resumes execution via `KVM_RUN`. This enables fast pause/resume cycles for debugging, migration, or resource management.

### Teardown

When a sandbox completes execution, CubeHypervisor halts the VM and releases all resources, including memory mappings and file descriptors, ensuring clean isolation boundary termination.

## Hardware-Level Isolation Mechanisms

CubeHypervisor leverages **hardware virtualization extensions** (Intel VT-x or AMD-V) to enforce strict isolation between MicroVMs and the host system. Each MicroVM operates in its own virtual address space protected by the CPU's memory management unit virtualization features.

The isolation guarantees include:

- **Memory Isolation**: Each VM runs in a separate address space preventing memory leaks or unauthorized access between VMs
- **Minimal Attack Surface**: The hypervisor only exposes essential KVM ioctls required for VM lifecycle management, reducing the code surface vulnerable to exploitation
- **Snapshot Integrity**: Snapshots are written atomically and can be cryptographically signed by higher-level CubeSandbox components to detect tampering

## Integration with CubeSandbox

CubeHypervisor exposes a public Go API in [`api.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/api.go) specifically designed for CubeSandbox's **Cubelet** component. This API enables:

- Launching sandboxes as MicroVMs via the hypervisor interface
- Pausing sandboxes by triggering snapshot operations followed by VM shutdown
- Resuming sandboxes by restoring snapshots and restarting VMs
- Performing incremental snapshots for rapid checkpoint/restore during debugging or migration

The tight coupling between Cubelet and CubeHypervisor allows CubeSandbox to provide a container-like developer experience while maintaining hardware-level isolation guarantees.

## Practical Implementation Example

The following Go code demonstrates how CubeHypervisor creates and runs a MicroVM using the public API:

```go
package main

import (
    "github.com/TencentCloud/cube-hypervisor"
    "log"
)

func main() {
    // Initialise the hypervisor (opens /dev/kvm)
    h, err := hypervisor.New()
    if err != nil {
        log.Fatalf("hypervisor init: %v", err)
    }
    defer h.Close()

    // Create a MicroVM with 16 MiB RAM and a tiny kernel image
    vm, err := h.NewVM(hypervisor.VMConfig{
        MemorySize: 16 << 20, 
        KernelPath: "./vmlinuz",
    })
    if err != nil {
        log.Fatalf("create vm: %v", err)
    }
    defer vm.Close()

    // Boot the VM – blocks until the VM exits or is paused
    if err := vm.Run(); err != nil {
        log.Fatalf("run vm: %v", err)
    }
}

```

This example initializes the KVM context, creates a 16 MiB MicroVM with a minimal kernel, and boots the VM using the `Run()` method.

## Summary

- CubeHypervisor is a lightweight KVM-based hypervisor that creates MicroVMs with ~10 MiB memory footprints for secure, isolated execution.
- The hypervisor manages VM lifecycle through [`hypervisor.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/hypervisor.go) (creation), [`vm.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/vm.go) (boot and vCPU management), and [`snapshot.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/snapshot.go) (checkpoint/restore).
- Hardware-level isolation is achieved via CPU virtualization extensions (VT-x/AMD-V) and the KVM driver, with each VM operating in an isolated address space.
- Incremental snapshots using dirty page tracking enable fast pause/resume operations for debugging and migration scenarios.
- The [`api.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/api.go) file provides a Go API consumed by CubeSandbox's Cubelet component, bridging high-level sandbox management with low-level KVM virtualization.

## Frequently Asked Questions

### How does CubeHypervisor differ from traditional hypervisors like QEMU?

Unlike QEMU, which provides full device emulation and supports a wide range of hardware configurations, CubeHypervisor is purpose-built for minimal MicroVMs. It strips away unnecessary emulation layers, uses direct KVM ioctls in [`hypervisor.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/hypervisor.go) for VM management, and focuses on fast startup times (~10 MiB memory footprints) rather than hardware compatibility. This minimal approach reduces the attack surface and resource overhead while maintaining hardware-level isolation.

### What files handle MicroVM snapshot and restore functionality?

The [`snapshot.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/snapshot.go) file in the TencentCloud/cube-hypervisor repository implements memory snapshot and incremental diff handling. It uses `KVM_GET_MEMORY_REGION` to capture VM memory states and tracks dirty pages to create efficient incremental snapshots. The restore process memory-maps saved pages back into the guest physical address space before resuming execution with `KVM_RUN`.

### How does CubeHypervisor ensure security between multiple MicroVMs?

CubeHypervisor enforces **hardware-level isolation** by leveraging the CPU's virtualization extensions (Intel VT-x or AMD-V) through the Linux KVM driver. Each MicroVM runs in its own isolated virtual address space managed by the hardware MMU, preventing memory leaks or unauthorized access between VMs. Additionally, the hypervisor minimizes its attack surface by exposing only essential KVM ioctls required for VM lifecycle operations.

### Can CubeHypervisor run standalone without CubeSandbox?

While CubeHypervisor can technically operate independently using its public Go API exposed in [`api.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/api.go), it is designed specifically as a submodule dependency for CubeSandbox. The hypervisor's API is tailored for CubeSandbox's Cubelet component, which manages sandbox lifecycle operations. Standalone usage would require implementing custom orchestration logic similar to what Cubelet provides.