# CubeHypervisor Architecture: How It Manages KVM MicroVMs in CubeSandbox

> Explore CubeHypervisor architecture and how it manages KVM microVMs. Discover its thin, hypervisor-agnostic abstraction layer for unified control over lightweight VMs in CubeSandbox.

- Repository: [Tencent Cloud/CubeSandbox](https://github.com/TencentCloud/CubeSandbox)
- Tags: architecture
- Published: 2026-07-15

---

**CubeHypervisor is a thin, hypervisor-agnostic abstraction layer that exposes generic `Hypervisor` and `Vm` traits over Linux KVM, enabling CubeSandbox to create and control lightweight micro-virtual machines through a unified Rust interface while maintaining direct control over vCPU file descriptors, memory mapping, and virtio device models.**

In the TencentCloud CubeSandbox ecosystem, the CubeHypervisor serves as the critical bridge between container-style isolation and hardware virtualization. Unlike monolithic hypervisors, this Rust-based virtualization layer decouples the underlying KVM (Kernel-based Virtual Machine) implementation from higher-level orchestration, allowing secure sandbox environments to boot in milliseconds with minimal device overhead.

## Core Architectural Components

The CubeHypervisor architecture follows a trait-based design pattern that separates interface definitions from concrete implementations.

- **Hypervisor trait** – Defined in [`hypervisor/src/hypervisor.rs`](https://github.com/TencentCloud/CubeSandbox/blob/main/hypervisor/src/hypervisor.rs), this generic API specifies methods for VM creation (`create_vm`), CPUID queries (`get_cpuid`), and extension validation (`check_required_extensions`). Any concrete hypervisor implementation, whether KVM or alternative backends, must fulfill this contract.

- **KVM-based implementation** – The default backend resides in [`hypervisor/vmm/src/vm.rs`](https://github.com/TencentCloud/CubeSandbox/blob/main/hypervisor/vmm/src/vm.rs) (VM lifecycle management) and [`hypervisor/vmm/src/vm_config.rs`](https://github.com/TencentCloud/CubeSandbox/blob/main/hypervisor/vmm/src/vm_config.rs) (configuration structures). These modules directly interact with the Linux KVM kernel module via `/dev/kvm` using raw ioctls.

- **VM abstraction** – The `Vm` trait represents a running micro-VM. Concrete implementations in [`hypervisor/vmm/src/vm.rs`](https://github.com/TencentCloud/CubeSandbox/blob/main/hypervisor/vmm/src/vm.rs) hold KVM file descriptors, manage vCPU state, handle guest memory layout, and coordinate device models.

- **Device models** – Minimal virtio implementations (virtio-blk, virtio-net, serial console) reside within `hypervisor/vmm/src/*`. These provide just enough paravirtualized I/O for sandboxed workloads without emulating legacy PC hardware.

- **CubeShim** – Located at [`CubeShim/shim/src/hypervisor/cube_hypervisor.rs`](https://github.com/TencentCloud/CubeSandbox/blob/main/CubeShim/shim/src/hypervisor/cube_hypervisor.rs), this thin shim translates CubeSandbox runtime requirements into hypervisor calls, setting up Linux namespaces (PID, network, mount) before launching the micro-VM.

- **API definition** – The OpenAPI specification in [`hypervisor/vmm/src/api/openapi/cloud-hypervisor.yaml`](https://github.com/TencentCloud/CubeSandbox/blob/main/hypervisor/vmm/src/api/openapi/cloud-hypervisor.yaml) documents the JSON-RPC/REST interface that SDKs and CLI tools use to communicate with the VMM (Virtual Machine Monitor).

## KVM MicroVM Lifecycle Management

CubeHypervisor manages micro-VMs through a rigorous seven-phase lifecycle that ensures hardware compatibility and resource isolation.

1. **Availability Verification** – Before initialization, the hypervisor confirms `/dev/kvm` exists and validates the kernel API version against expected ranges, returning `HypervisorError::HypervisorAvailableCheck` on mismatch.

2. **VM Structure Allocation** – Calling `Hypervisor::create_vm` allocates a new KVM file descriptor via `KVM_CREATE_VM`, instantiates the concrete `Vm` struct, and reserves guest physical memory using anonymous `mmap` regions.

3. **CPU Feature Negotiation** – On x86_64 platforms, the hypervisor queries host capabilities via `KVM_GET_SUPPORTED_CPUID`, filters the CPUID entries through `get_cpuid`, and validates required extensions such as `KVM_CAP_USER_MEMORY` against the hardware using `check_required_extensions`.

4. **vCPU Initialization** – For each requested virtual CPU, the system:
   - Creates a vCPU file descriptor with `KVM_CREATE_VCPU`
   - Loads initial register states and guest entry point addresses
   - Establishes virtual interrupt controllers and clock sources

5. **Memory and Device Attachment** – Guest RAM backs onto either anonymous memory or file-based mappings. The configuration defined in [`vm_config.rs`](https://github.com/TencentCloud/CubeSandbox/blob/main/vm_config.rs) attaches virtio-blk, virtio-net, and console devices via MMIO or PCI transport, depending on the platform.

6. **Execution Entry** – The CubeShim invokes `Vm::run`, which enters the `KVM_RUN` ioctl loop. This loop handles VM exits—such as I/O port accesses, MSR reads, or virtio device notifications—dispatching them to the appropriate device emulators in user space.

7. **Resource Cleanup** – Upon termination, the hypervisor tracks state transitions (running, paused, stopped) and provides cleanup APIs that close KVM file descriptors, unmap guest memory, and delete temporary sandbox resources.

## Key Architectural Design Goals

The CubeHypervisor prioritizes three fundamental principles that distinguish it from traditional virtualization stacks.

**Hypervisor-agnostic interface** – By strictly separating the `Hypervisor` and `Vm` traits from KVM-specific internals, CubeSandbox can theoretically extend support to alternative backends like Firecracker or Cloud-Hypervisor without modifying higher-level SDKs or orchestration logic.

**Lightweight isolation** – Micro-VMs intentionally expose only the minimal device set required for sandboxed execution. This reduction in emulated hardware surface area directly translates to faster cold starts and fewer potential attack vectors compared to full-QEMU emulation.

**Hardware extensibility** – Conditional compilation flags such as `#[cfg(feature = "tdx")]` allow the hypervisor to integrate Intel TDX (Trust Domain Extensions) support alongside architecture-specific CPUID handling, ensuring the stack evolves with new silicon capabilities while maintaining backward compatibility.

## Implementation Examples

The following Rust patterns demonstrate how CubeSandbox instantiates and configures micro-VMs through the hypervisor abstraction.

**Creating and launching a micro-VM:**

```rust
use hypervisor::Hypervisor;
use hypervisor::hypervisor::HypervisorError;
use std::sync::Arc;

// Factory function returns the KVM-backed implementation
let hv: Arc<dyn Hypervisor> = hypervisor::new_kvm_hypervisor()?;
let vm: Arc<dyn Vm> = hv.create_vm()?;          // Issues KVM_CREATE_VM
vm.configure(&vm_config)?;                     // Applies memory layout and device tree
vm.start()?;                                   // Enters KVM_RUN loop

```

**Querying host CPUID support for guest compatibility:**

```rust
let cpuid = hv.get_cpuid()?; 
for entry in cpuid {
    println!("leaf {:#x}, subleaf {:#x}", entry.function, entry.index);
}

```

## Summary

- CubeHypervisor acts as a Rust trait-based abstraction over Linux KVM, defined primarily in [`hypervisor/src/hypervisor.rs`](https://github.com/TencentCloud/CubeSandbox/blob/main/hypervisor/src/hypervisor.rs) and implemented in [`hypervisor/vmm/src/vm.rs`](https://github.com/TencentCloud/CubeSandbox/blob/main/hypervisor/vmm/src/vm.rs).

- The architecture separates generic hypervisor operations from concrete KVM ioctls, enabling potential multi-backend support while maintaining tight control over `KVM_CREATE_VCPU`, `KVM_RUN`, and memory management.

- Micro-VMs follow a strict lifecycle from availability checks through resource cleanup, with configuration centralized in [`vm_config.rs`](https://github.com/TencentCloud/CubeSandbox/blob/main/vm_config.rs) and device models kept intentionally minimal.

- Integration with CubeSandbox occurs through [`CubeShim/shim/src/hypervisor/cube_hypervisor.rs`](https://github.com/TencentCloud/CubeSandbox/blob/main/CubeShim/shim/src/hypervisor/cube_hypervisor.rs), which bridges Linux namespaces with the virtual machine monitor.

## Frequently Asked Questions

### What differentiates CubeHypervisor from Firecracker or Cloud-Hypervisor?

While all three target lightweight virtualization, CubeHypervisor specifically serves as a hypervisor-agnostic abstraction layer within the CubeSandbox ecosystem. Unlike Firecracker, which is tightly coupled to its own VMM implementation, CubeHypervisor defines generic `Hypervisor` and `Vm` traits in [`hypervisor/src/hypervisor.rs`](https://github.com/TencentCloud/CubeSandbox/blob/main/hypervisor/src/hypervisor.rs) that could theoretically support multiple backends. The OpenAPI specification at [`hypervisor/vmm/src/api/openapi/cloud-hypervisor.yaml`](https://github.com/TencentCloud/CubeSandbox/blob/main/hypervisor/vmm/src/api/openapi/cloud-hypervisor.yaml) provides standardized REST/JSON-RPC interfaces, whereas Firecracker exposes a specific Unix socket API.

### How does CubeHypervisor ensure CPU compatibility between host and guest?

During the initialization phase, the hypervisor executes `get_cpuid` to retrieve host capabilities via `KVM_GET_SUPPORTED_CPUID`, then filters these entries to present only supported features to the guest. The `check_required_extensions` method validates that the host kernel exposes necessary capabilities like `KVM_CAP_USER_MEMORY` before VM creation proceeds. This prevents runtime faults from incompatible CPU feature flags.

### Can CubeHypervisor run on non-KVM hypervisors?

The architecture supports this through its trait-based design. While the current implementation in [`hypervisor/vmm/src/vm.rs`](https://github.com/TencentCloud/CubeSandbox/blob/main/hypervisor/vmm/src/vm.rs) specifically targets Linux KVM via `/dev/kvm`, the generic `Hypervisor` trait defined in [`hypervisor/src/hypervisor.rs`](https://github.com/TencentCloud/CubeSandbox/blob/main/hypervisor/src/hypervisor.rs) contains no KVM-specific dependencies. Adding support for Microsoft Hyper-V, Xen, or userspace VMMs would require implementing these traits for the respective backend without altering the CubeSandbox SDKs.

### Where is the virtual hardware configuration defined for each micro-VM?

Guest configuration—including memory layout, vCPU count, and virtio device attachments—is structured in [`hypervisor/vmm/src/vm_config.rs`](https://github.com/TencentCloud/CubeSandbox/blob/main/hypervisor/vmm/src/vm_config.rs). This module defines the serializable structures that parse the OpenAPI spec from [`hypervisor/vmm/src/api/openapi/cloud-hypervisor.yaml`](https://github.com/TencentCloud/CubeSandbox/blob/main/hypervisor/vmm/src/api/openapi/cloud-hypervisor.yaml), allowing CLI tools and the CubeShim to pass consistent configuration parameters to the `Vm::configure` method before the `KVM_RUN` loop begins.