# CubeShim Containerd Shim v2 Architecture: How CubeSandbox Bridges Containers and MicroVMs

> Discover the CubeShim containerd shim v2 architecture. Learn how CubeSandbox bridges containers and microVMs using Rust and ttrpc for efficient communication.

- Repository: [Tencent Cloud/CubeSandbox](https://github.com/TencentCloud/CubeSandbox)
- Tags: architecture
- Published: 2026-07-03

---

**CubeShim implements the containerd shim v2 specification as a Rust-based bridge that translates container lifecycle commands into MicroVM operations, communicating with an in-guest cube-agent via ttrpc over vsock.**

CubeShim is the critical integration point in the TencentCloud/CubeSandbox project that allows containerd to manage sandboxed MicroVMs using standard container semantics. By implementing the containerd shim v2 API, CubeShim enables container orchestrators to treat lightweight virtual machines as regular containers without modifying the upper stack. This article examines the architecture of the CubeShim containerd shim v2 implementation, tracing how it handles container lifecycle events from the initial RPC to VM process execution.

## High-Level Data Flow and Architecture

The CubeShim architecture follows a three-tier communication model that isolates the container engine from the guest runtime:

```

containerd
   │   (Shim v2 API)
   ▼
containerd-shim-cube-rs  ← CubeShim (this binary)
   │   (ttrpc)
   ▼
cube-agent (in-VM)
   │
   ▼
processes inside the MicroVM

```

**containerd** issues standard Shim v2 calls such as `Create`, `Start`, `Exec`, and `Kill`. **CubeShim** runs as the binary `containerd-shim-cube-rs` and implements these calls. For each request, it communicates with the **cube-agent** inside the guest VM via **ttrpc** over a vsock connection. The agent then forwards the request to the actual process runtime inside the MicroVM.

This architecture is documented in the Mermaid diagram within [`docs/architecture/overview.md`](https://github.com/TencentCloud/CubeSandbox/blob/main/docs/architecture/overview.md) (lines 25-27).

## Core Components of the CubeShim Implementation

### Shim Process and Entry Point

The shim binary entry point resides in [`CubeShim/shim/src/main.rs`](https://github.com/TencentCloud/CubeSandbox/blob/main/CubeShim/shim/src/main.rs). This component boots a **Tokio** async runtime, parses command-line flags, and invokes `shim_run::<Service>(...)` to register the service implementation with the containerd-shim-rs framework. This entry point is responsible for initializing the long-running process that containerd will communicate with throughout the container lifecycle.

### Service Implementation Layer

The concrete RPC handlers live in `CubeShim/shim/src/service/`, specifically within [`srv.rs`](https://github.com/TencentCloud/CubeSandbox/blob/main/srv.rs) and [`task_srv.rs`](https://github.com/TencentCloud/CubeSandbox/blob/main/task_srv.rs). These files implement the **Service** struct that handles Shim v2 API methods including `create`, `start`, `exec`, `kill`, and `delete`. Each handler forwards requests to the in-VM **cube-agent** via a **ttrpc** client, translating containerd's expectations into VM-specific operations.

### Hypervisor Integration

VM lifecycle management is abstracted in [`CubeShim/shim/src/hypervisor/cube_hypervisor.rs`](https://github.com/TencentCloud/CubeSandbox/blob/main/CubeShim/shim/src/hypervisor/cube_hypervisor.rs). This module wraps **CubeHypervisor** (built on RustVMM and KVM) to handle MicroVM creation, snapshot, restore, pause, and resume operations. When the shim receives a create request, it asks the hypervisor to instantiate a MicroVM, then extracts the vsock endpoint to hand off to the agent.

### Sandbox Resource Preparation

Before VM launch, the shim prepares necessary resources in `CubeShim/shim/src/container/`. Files like [`rootfs.rs`](https://github.com/TencentCloud/CubeSandbox/blob/main/rootfs.rs) and [`exec.rs`](https://github.com/TencentCloud/CubeSandbox/blob/main/exec.rs) pre-create the root filesystem, memory file, and kernel image that the VM will use. These resources are supplied to the hypervisor during the `Create` RPC handling.

### Logging and Statistics

Operational telemetry is managed through [`CubeShim/shim/src/log/stat_defer.rs`](https://github.com/TencentCloud/CubeSandbox/blob/main/CubeShim/shim/src/log/stat_defer.rs). This component writes request logs and state transitions to `/data/log/CubeShim/`, ensuring that shim operations remain observable without affecting the containerd daemon's performance.

### Common Utilities

Shared types, constants (such as `SHIM_VERSION`), and helper functions reside in `CubeShim/shim/src/common/`. These utilities provide the foundational types used across the shim's service, hypervisor, and container modules.

## Container Lifecycle Implementation

### Create and Start Sequence

The interaction between containerd and CubeShim follows a precise sequence when spawning a sandboxed workload:

1. **containerd** spawns `containerd-shim-cube-rs` with the sandbox ID and bundle path.
2. The shim's [`main.rs`](https://github.com/TencentCloud/CubeSandbox/blob/main/main.rs) creates a multi-threaded Tokio runtime and calls `shim_run::<Service>(...)`, registering the service implementation.
3. The **Create** RPC arrives at `Service::create` in [`srv.rs`](https://github.com/TencentCloud/CubeSandbox/blob/main/srv.rs). This handler reads the sandbox bundle (rootfs, kernel, and memory file), then calls `cube_hypervisor::launch_vm` to instantiate the KVM MicroVM and start the cube-agent via vsock.
4. The **Start** RPC triggers `Service::start`, which forwards an `Exec` request over ttrpc to the agent. The agent finally executes the user's entrypoint process inside the MicroVM.
5. The shim streams the VM's stdin, stdout, and stderr back to containerd, presenting the MicroVM's output as standard container logs.

This orchestration is managed by the async service defined in [`CubeShim/shim/src/service/srv.rs`](https://github.com/TencentCloud/CubeSandbox/blob/main/CubeShim/shim/src/service/srv.rs) and the state-tracking helpers in [`task_srv.rs`](https://github.com/TencentCloud/CubeSandbox/blob/main/task_srv.rs).

## Containerd Integration and Registration

To enable containerd to route requests to CubeShim, the runtime must be registered in [`/etc/containerd/config.toml`](https://github.com/TencentCloud/CubeSandbox/blob/main//etc/containerd/config.toml). As documented in [`CubeShim/README.md`](https://github.com/TencentCloud/CubeSandbox/blob/main/CubeShim/README.md) (lines 81-87), the configuration requires:

```toml
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.cube]
  runtime_type = "io.containerd.cube.v2"

```

The `containerd-shim-cube-rs` binary must be available on the host `$PATH`. Once configured, containerd invokes the shim whenever a sandbox using the `cube` runtime is created.

## Why Shim v2 Architecture Matters for CubeSandbox

The Shim v2 model provides three critical advantages for CubeSandbox:

- **Process Isolation:** The shim remains alive after the initial containerd fork, decoupling the MicroVM's lifetime from the containerd process that launched it. This prevents containerd restarts from affecting running workloads.
- **Auto-Pause and Resume:** CubeShim can receive pause and resume requests from the control plane (CubeMaster) and invoke the hypervisor's snapshot and restore paths without involving containerd directly.
- **Stateless Containerd Integration:** By implementing the minimal subset of the Shim v2 API required for CubeSandbox, the shim keeps containerd's view of the sandbox simple while performing heavy lifting inside the VM.

## Configuration and Usage Examples

### Containerd Runtime Configuration

To enable the cube runtime, add the following to your containerd configuration:

```toml

# /etc/containerd/config.toml

[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.cube]
  runtime_type = "io.containerd.cube.v2"
  # Optional: explicit path if not on $PATH

  #runtime_path = "/usr/local/bin/containerd-shim-cube-rs"

```

### Creating a Sandbox via Containerd

Once configured, create a sandboxed container using the standard containerd CLI:

```bash

# Trigger containerd to launch containerd-shim-cube-rs

ctr run --runtime=cube \
    docker.io/library/ubuntu:22.04 \
    my-sandbox-id \
    /bin/bash -c "echo Hello from MicroVM"

```

This command initiates the full lifecycle sequence, from shim spawning to VM process execution.

### Minimal Shim Client Example

For testing or custom integrations, the shim can be invoked programmatically:

```rust
use containerd_shim::asynchronous::run as shim_run;
use containerd_shim_cube_rs::service::Service;

#[tokio::main]
async fn main() {
    // Register the service under the cube runtime name
    shim_run::<Service>("io.containerd.cube.rs", None).await;
}

```

This pattern mirrors the actual entry point in [`CubeShim/shim/src/main.rs`](https://github.com/TencentCloud/CubeSandbox/blob/main/CubeShim/shim/src/main.rs) (lines 49-55).

## Summary

- CubeShim implements the **containerd shim v2 API** in Rust, acting as a bridge between containerd and MicroVMs.
- The architecture uses **ttrpc over vsock** to communicate with the **cube-agent** inside the guest, forwarding container lifecycle commands.
- Core components are organized into modular directories: `service/` for RPC handlers, `hypervisor/` for VM management, and `container/` for resource preparation.
- The **Create** and **Start** sequence involves hypervisor calls to `cube_hypervisor::launch_vm` followed by agent communication to execute processes.
- Registration requires setting `runtime_type = "io.containerd.cube.v2"` in containerd's configuration.

## Frequently Asked Questions

### What is the role of cube-agent in the CubeShim architecture?

The **cube-agent** is a process running inside the MicroVM that receives translated container commands from CubeShim via ttrpc. It acts as the final executor, spawning processes inside the guest and managing their I/O, effectively serving as the container runtime within the VM boundary.

### How does CubeShim communicate with the MicroVM?

CubeShim communicates with the MicroVM using **ttrpc** (a lightweight RPC protocol) transported over **vsock** (virtual socket). This mechanism allows the host-side shim to send commands to the in-guest cube-agent without requiring network configuration, providing a secure, low-latency control channel.

### What hypervisor does CubeShim use for VM management?

CubeShim integrates with **CubeHypervisor**, a RustVMM-based virtualization layer that uses KVM. The integration code in [`CubeShim/shim/src/hypervisor/cube_hypervisor.rs`](https://github.com/TencentCloud/CubeSandbox/blob/main/CubeShim/shim/src/hypervisor/cube_hypervisor.rs) handles VM launch, snapshot, restore, and pause/resume operations, abstracting the underlying KVM interactions from the shim service layer.

### Why does CubeShim implement the shim v2 API instead of v1?

The **shim v2** specification provides a lightweight, long-running process model that isolates the container runtime from the containerd daemon. This allows CubeShim to manage MicroVM lifecycles independently, supporting features like auto-pause/resume and snapshotting without requiring containerd to maintain state or handle VM-specific complexity.