CubeShim in the CubeSandbox Architecture: The containerd Shim v2 Bridge to MicroVMs

CubeShim is the containerd Shim v2 implementation written in Rust that bridges the container runtime (containerd) and the KVM-based MicroVM lifecycle in CubeSandbox, enabling standard container APIs to launch hardware-isolated workloads.

CubeSandbox is an open-source sandbox runtime from Tencent Cloud that uses MicroVMs for container isolation. At its heart lies CubeShim, the critical adapter that translates containerd's high-level container operations into low-level virtual machine management commands. This component allows CubeSandbox to appear as a standard container runtime to containerd while actually running workloads inside fast-booting, hardware-isolated KVM MicroVMs.

How CubeShim Fits in the Architecture

CubeShim operates as the middle layer in a four-tier architecture that abstracts virtualization complexity behind familiar container interfaces.

Control Plane (containerd)

The containerd daemon issues sandbox creation requests via the Shim v2 API. It invokes the CubeShim binary (containerd-shim-cube-rs) without needing to understand underlying virtualization details.

Shim Layer (CubeShim)

This Rust-based service implements the Shim v2 interface expected by containerd. It translates API calls such as Create, Start, and Exec into specific actions performed by the hypervisor, including VM creation, lifecycle management, and resource allocation.

Data Plane (CubeHypervisor)

Built on RustVMM and KVM, this layer manages the actual MicroVM including vCPU scheduling, memory allocation, and virtio device configuration. CubeHypervisor receives launch and restore commands from CubeShim and reports VM readiness back to it.

In-VM Agent (cube-agent)

Running inside each MicroVM, this agent handles container-level syscalls over ttrpc via vsock. CubeShim forwards execution requests, signals, and I/O streams to this agent to interact with the actual container process.

According to the architecture documentation in docs/architecture/overview.md, CubeShim is explicitly designed to "bridge the container runtime abstraction and the actual MicroVM"【/cache/repos/github.com/TencentCloud/CubeSandbox/master/docs/architecture/overview.md#L74-L78】.

Core Responsibilities of CubeShim

CubeShim manages the entire lifecycle of a sandboxed container through five critical functions.

VM Resource Preparation

Before booting, CubeShim allocates and prepares the root filesystem, memory image, and kernel binary, passing these resources to the CubeHypervisor for VM construction.

VM Boot and Restore

CubeShim invokes CubeHypervisor to either create a fresh sandbox or restore one from a snapshot. This capability enables fast cold-start and auto-pause/resume functionality critical for serverless workloads.

ttrpc Communication

CubeShim establishes a vsock channel to the in-VM cube-agent and forwards container lifecycle RPCs including Create, Start, Exec, Kill, and Delete.

I/O and Signal Proxying

The shim relays standard input/output streams and POSIX signals between the host process and the container process running inside the VM, ensuring seamless interaction.

State Reporting

CubeShim emits Shim v2 events such as running, paused, and exited to containerd, allowing the orchestrator to track sandbox status without understanding virtualization states.

Implementation Details

Registering CubeShim with containerd

To enable CubeShim as a runtime, configure containerd with the following TOML snippet found in the CubeShim documentation:

# /etc/containerd/config.toml

[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.cube]
  runtime_type = "io.containerd.cube.v2"

This registration tells containerd to invoke the containerd-shim-cube-rs binary when users request the cube runtime【/cache/repos/github.com/TencentCloud/CubeSandbox/master/CubeShim/README.md#L81-L87】.

Launching a Sandbox

Once registered, create a sandbox using the standard containerd CLI. CubeShim handles the virtualization transparently:

# Create a sandbox using the Cube runtime

ctr run --rm \
    --runtime=cube \
    docker.io/library/ubuntu:22.04 \
    my-sandbox-id \
    /bin/bash -c "echo Hello from inside the MicroVM"

When this command executes, CubeShim receives the Create request, coordinates resource allocation through Cubelet, and calls CubeHypervisor to start the VM. The command returns only after the VM reports readiness via the agent.

Shim Service Entry Point

The main entry point in CubeShim/shim/src/main.rs initializes the asynchronous Shim v2 service:

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Initialize logging, parse args, and start the shim service.
    let svc = shim::service::ShimService::new().await?;
    svc.serve().await?;
    Ok(())
}

This Tokio-based async runtime manages the long-lived shim process that containerd uses to control the sandbox lifecycle【/cache/repos/github.com/TencentCloud/CubeSandbox/master/CubeShim/shim/src/main.rs#L1-L10】.

Agent Communication

CubeShim communicates with the in-VM agent via ttrpc over vsock. A simplified interaction pattern from the shim API documentation appears as:

let mut client = TtrpcClient::connect("/run/vsock/guest/agent.sock").await?;
let exec_req = ExecRequest { ... };
let resp = client.exec(exec_req).await?;

This channel carries all container operations from the host shim to the guest agent【/cache/repos/github.com/TencentCloud/CubeSandbox/master/CubeShim/docs/shimapi/README.md】.

Key Source Files

Understanding CubeShim requires examining these specific locations in the TencentCloud/CubeSandbox repository:

• docs/architecture/overview.md: Contains the high-level system architecture description explaining CubeShim's position between containerd and the hypervisor.
• CubeShim/README.md: Documents the shim's responsibilities and containerd integration points.
• CubeShim/shim/src/main.rs: The entrypoint implementing the shim process initialization and service startup.
• CubeShim/shim/src/service/srv.rs: Implements the Shim v2 RPC handlers including Create, Start, Exec, and Delete.
• CubeShim/protoc/protos/oci.proto: Defines the protocol buffer specifications for container-runtime interactions inherited from the Kata Containers project.

Summary

• CubeShim is the containerd Shim v2 implementation that enables CubeSandbox to expose standard container APIs while running workloads in KVM MicroVMs.
• It acts as a translation layer between containerd's high-level operations and the low-level CubeHypervisor virtualization interface.
• Written in Rust using Tokio for async operations, it manages VM lifecycle, resource preparation, and snapshot restore operations.
• Communication with container processes occurs via ttrpc over vsock to the in-VM cube-agent.
• Configuration requires only a runtime type registration in containerd's config.toml, making integration transparent to Kubernetes and other orchestrators.

Frequently Asked Questions

How does CubeShim differ from CubeHypervisor?

CubeShim implements the containerd Shim v2 API and manages the control plane logic, while CubeHypervisor handles the actual virtualization using RustVMM and KVM. CubeShim translates container operations into VM management commands but does not directly manipulate vCPUs or memory; it delegates hardware virtualization to CubeHypervisor.

What protocol does CubeShim use to communicate with containers inside the MicroVM?

CubeShim uses ttrpc (a lightweight RPC framework) over vsock (virtual socket) to communicate with the cube-agent running inside each MicroVM. This channel forwards Exec, Kill, and I/O requests while maintaining isolation between the host and guest.

Can CubeShim be used with existing Kubernetes clusters?

Yes. Because CubeShim implements the standard Shim v2 interface expected by containerd, it integrates seamlessly with Kubernetes without requiring changes to the orchestrator. Administrators simply register the cube runtime in containerd's configuration, then specify runtimeClassName: cube in Pod specifications.

Why is CubeShim implemented in Rust?

CubeShim is written in Rust to leverage memory safety guarantees and high-performance async I/O through Tokio. This aligns with the broader CubeSandbox architecture, which uses RustVMM for the hypervisor layer, ensuring consistent memory management and safe concurrency across the virtualization stack.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}