How CubeShim Implements the containerd Shim v2 API: A Deep Dive into the Rust Implementation

CubeShim implements the containerd Shim v2 API by registering as a Rust-based ttrpc server that bridges containerd’s gRPC calls to Micro-VM operations, using the containerd_shim crate’s Shim and Task traits in CubeShim/shim/src.

CubeShim serves as the critical glue within the TencentCloud/CubeSandbox repository that allows Micro-VMs managed by Cube Hypervisor to appear as standard containers to containerd. By implementing the containerd Shim v2 API, CubeShim translates high-level container lifecycle commands—such as create, start, and delete—into low-level VM operations, enabling Kubernetes to orchestrate lightweight virtual machines using the familiar container interface.

Registration and Entry Point

The shim lifecycle begins in CubeShim/shim/src/main.rs, where the entry point constructs a Tokio runtime and parses command-line flags using containerd_shim::parse. The critical registration happens at lines 54–55:

runtime.block_on(shim_run::<Service>("io.containerd.cube.rs", Some(c)));

The string io.containerd.cube.rs is the runtime identifier that containerd uses to locate the shim binary. This call hands control to the containerd_shim framework, instructing it to use the Service type as the concrete implementation. The main.rs file also handles the runtime info handshake (-info flag) in handle_runtime_info_request (lines 74–89) by copying stdin to /dev/null and flushing stdout, returning an empty protobuf to satisfy containerd v2 requirements.

The Service Shim Implementation

The Service struct defined in CubeShim/shim/src/service/srv.rs (lines 18–34) implements the containerd_shim::Shim trait, forming the core of the Shim v2 contract. Each method fulfills a specific lifecycle requirement:

new (lines 30–37): Captures the sandbox ID, namespace, and creates an ExitSignal to manage the shim's lifespan.

start_shim (lines 39–46): Spawns the ttrpc server that containerd talks to. It writes the socket address to $DATA_LOG_DIR/CubeShim/address, enabling containerd to establish the communication channel.

delete_shim (lines 56–71): Handles cleanup by sending SIGTERM and SIGKILL to the shim PID, removing the socket file, and calling Utils::clean_sandbox_resource to release sandbox resources.

wait (lines 74–76): Blocks on the ExitSignal, keeping the shim process alive until containerd signals termination.

create_task_service (lines 78–86): Instantiates a TaskService (the concrete container-task handler) and passes a RemotePublisher to enable event streaming back to containerd.

TaskService and Container Lifecycle

While Service manages the shim itself, TaskService in CubeShim/shim/src/service/task_srv.rs implements the containerd_shim::task::Task trait (lines 84–86) to handle container-level operations. This is where CubeShim translates containerd requests into CubeSandbox operations.

Create, Start, and Exec Operations

The create method (lines 87–131) loads the OCI runtime specification, initializes the sandbox if necessary, and creates the VM via CubeHypervisor. The sandbox creation flow (lines 131–166) restores the Micro-VM from a memory snapshot for O(1) startup time, returning the VM PID in the response so containerd can track the process. The start method simply resumes the VM if paused and forwards the containerd event, while exec, resize, and update forward to the internal container manager handling vsock communication.

Event Publishing

All task methods send asynchronous events back to containerd through a RemotePublisher using the tx_event helper (lines 77–81). This continuous stream satisfies containerd’s expectation for events like TaskCreate, TaskStart, and TaskExit, ensuring proper state synchronization.

Micro-VM Backend Integration

The sandbox logic resides in CubeShim/shim/src/sandbox/. The sb.rs file owns the VM lifecycle, providing init, create_sandbox, pid, and paused methods. The hypervisor/cube_hypervisor.rs file wraps Rust-VMM (KVM) calls to launch or restore Micro-VMs.

When TaskService::create calls sandbox::SandBox::create_sandbox(), the shim leverages the "auto-pause" feature implemented in pmem.rs, which handles in-place memory snapshots. This allows CubeShim to restore a sandbox instantly without copying data, achieving millisecond-level container startup times.

Runtime Configuration and Discovery

To enable containerd discovery, configure /etc/containerd/config.toml:

[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.cube]
  runtime_type = "io.containerd.runc.v2"
  pod_annotations = ["io.kubernetes.cri.sandbox-image"]
  shim = "/usr/local/bin/containerd-shim-cube-rs"

The runtime_type points to the Shim v2 protocol, while the shim path specifies the compiled containerd-shim-cube-rs binary produced by CubeShim. Additionally, main.rs configures kernel-level resource limits in set_process (lines 91–140), setting core-dump filters and RLIMIT_NOFILE to prevent resource exhaustion when running many containers concurrently.

Summary

• CubeShim registers as the io.containerd.cube.rs runtime via shim_run::<Service> in main.rs.
• The Service struct in srv.rs implements the containerd_shim::Shim trait to manage shim lifecycle, socket creation, and cleanup.
• TaskService in task_srv.rs implements the Task trait to handle OCI container operations and stream events via RemotePublisher.
• Integration with Cube Hypervisor through sb.rs and cube_hypervisor.rs enables O(1) Micro-VM restoration from memory snapshots.
• The shim writes its ttrpc address to $DATA_LOG_DIR/CubeShim/address and handles the -info handshake for containerd v2 compatibility.

Frequently Asked Questions

What specific traits does CubeShim implement to satisfy the Shim v2 API?

CubeShim implements two primary traits from the containerd_shim crate: the Shim trait in CubeShim/shim/src/service/srv.rs (lines 18–34) for shim lifecycle management, and the Task trait in CubeShim/shim/src/service/task_srv.rs (lines 84–86) for container task operations. These implementations bridge containerd’s gRPC calls to the underlying CubeSandbox Micro-VM management.

How does CubeShim communicate its socket address back to containerd?

During the start_shim method in srv.rs (lines 39–46), CubeShim spawns a ttrpc server and writes the socket address to a file located at $DATA_LOG_DIR/CubeShim/address. Containerd reads this file to establish the communication channel with the running shim process.

What happens when containerd sends a delete request to CubeShim?

The delete_shim method in srv.rs (lines 56–71) handles cleanup by first sending SIGTERM and then SIGKILL to the shim PID. It then removes the socket file and invokes Utils::clean_sandbox_resource to ensure all sandbox resources are properly released before the process exits.

How does CubeShim handle the OCI runtime specification during container creation?

In task_srv.rs, the create method (lines 87–131) loads the OCI spec to configure the container environment. It then initializes the CubeSandbox and creates the VM via the CubeHypervisor (lines 131–166) before instantiating the container inside the sandbox, effectively treating the Micro-VM as the container execution environment.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}