How CubeShim Implements the containerd Shim v2 API for Micro-VM Integration

CubeShim implements the containerd Shim v2 API by exposing a Rust-based gRPC service that translates containerd lifecycle commands into Micro-VM operations via the Service and TaskService structs, enabling containerd to manage CubeSandbox Micro-VMs as standard OCI containers.

CubeShim serves as the bridge between containerd and the Cube Hypervisor, allowing Kubernetes workloads to run inside lightweight Micro-VMs. By implementing the containerd Shim v2 interface defined in the containerd_shim crate, CubeShim makes VM-based sandboxes appear as regular container runtimes to the containerd daemon.

Runtime Registration and Entry Point

The shim binary initializes in CubeShim/shim/src/main.rs by building a Tokio async runtime and parsing command-line flags via containerd_shim::parse. The critical registration happens when the runtime invokes:

runtime.block_on(shim_run::<Service>("io.containerd.cube.rs", Some(c)));

This call (found at [main.rs L54-L55]) registers the runtime name io.containerd.cube.rs with the containerd Shim v2 framework, instructing it to use the local Service type for all shim operations.

Containerd v2 occasionally queries the shim with the -info flag. CubeShim handles this handshake in handle_runtime_info_request ([main.rs L74-L89]), which drains stdin and returns an empty protobuf response to satisfy the protocol without heavy processing.

The Service Struct: Core Shim v2 Implementation

The Service struct defined in [srv.rs L18-L34] implements the containerd_shim::Shim trait, satisfying the four mandatory lifecycle methods required by the Shim v2 specification:

new ([srv.rs L30-L37])

• Captures the sandbox ID, namespace, and initializes an ExitSignal for graceful termination.

start_shim ([srv.rs L39-L46])

• Spawns the ttrpc server that containerd uses for communication.
• Writes the socket address to $DATA_LOG_DIR/CubeShim/address so containerd knows where to connect.

delete_shim ([srv.rs L56-L71])

• Handles cleanup by sending SIGTERM then SIGKILL to the shim PID.
• Removes the socket file and invokes Utils::clean_sandbox_resource to release VM resources.

wait ([srv.rs L74-L76])

• Blocks on the ExitSignal channel, keeping the shim process alive until containerd signals exit.

create_task_service ([srv.rs L78-L86])

• Instantiates TaskService for the sandbox, injecting a RemotePublisher to stream lifecycle events (TaskCreate, TaskStart, TaskExit) back to containerd.

TaskService: Container Lifecycle Management

TaskService implements the containerd_shim::task::Task trait ([task_srv.rs L84-L86]), mapping containerd's container-level RPCs to CubeSandbox operations. Each method converts OCI runtime requests into Micro-VM commands:

Create ([task_srv.rs L87-L131] and [task_srv.rs L131-L166])

• Loads the OCI specification and initializes the CubeSandbox if absent.
• Invokes CubeHypervisor to restore the VM from a memory snapshot (enabling O(1) startup via auto-pause/resume).
• Returns the VM PID to containerd for process tracking.

Start, Exec, Kill

• Start resumes the VM if paused and publishes the TaskStart event.
• Exec and Resize forward to the internal container manager over vsock channels.
• Kill propagates signals to the Micro-VM guest.

All operations emit events through the RemotePublisher (accessed via the tx_event helper at [task_srv.rs L77-L81]), maintaining the event stream contract required by containerd.

Sandbox and Hypervisor Integration

The actual VM lifecycle logic resides in CubeShim/shim/src/sandbox/:

• sb.rs – Owns sandbox state through methods like create_sandbox, init, pid, and paused, coordinating with the hypervisor layer.
• hypervisor/cube_hypervisor.rs – Wraps Rust-VMM/KVM calls to launch or restore Micro-VMs.
• pmem.rs – Manages in-place memory snapshots for automatic pause/resume, allowing instant container startup without data copying.

When TaskService::create calls sandbox::SandBox::create_sandbox(), the shim requests the hypervisor to restore the VM from a pre-existing memory snapshot rather than booting from scratch.

Containerd Configuration and Runtime Discovery

Containerd discovers CubeShim through configuration in /etc/containerd/config.toml:

[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.cube]
  runtime_type = "io.containerd.runc.v2"
  pod_annotations = ["io.kubernetes.cri.sandbox-image"]
  shim = "/usr/local/bin/containerd-shim-cube-rs"

The runtime_type signals Shim v2 protocol support, while the shim path points to the compiled CubeShim binary. When Kubernetes schedules a pod with runtime class cube, containerd executes this binary, which executes the initialization flow in main.rs, publishes its socket address, and awaits Task RPCs.

Summary

• CubeShim registers as io.containerd.cube.rs via shim_run in main.rs, implementing the containerd Shim v2 trait.
• The Service struct handles shim lifecycle (start, wait, delete) and address management in $DATA_LOG_DIR/CubeShim/address.
• TaskService translates containerd container operations into Micro-VM commands, supporting create, start, exec, and kill via the Cube Hypervisor.
• VM restoration from memory snapshots (managed in pmem.rs) enables O(1) container startup times.
• Containerd discovery requires registering the containerd-shim-cube-rs binary path in the containerd configuration TOML.

Frequently Asked Questions

What is the difference between Shim v2 and v1 in CubeShim?

CubeShim exclusively implements Shim v2, which uses a direct ttrpc connection between containerd and the shim process, rather than the older v1 architecture that required an additional containerd-shim process per container. This reduces memory overhead and improves startup latency for Micro-VM workloads.

How does CubeShim handle process cleanup when a sandbox is deleted?

When containerd calls delete, the delete_shim method ([srv.rs L56-L71]) sends SIGTERM followed by SIGKILL to the shim PID, removes the Unix socket file from $DATA_LOG_DIR/CubeShim/, and invokes Utils::clean_sandbox_resource to release KVM file descriptors and memory mappings managed by the Cube Hypervisor.

Where does CubeShim store runtime state and logs?

CubeShim writes its ttrpc socket address to $DATA_LOG_DIR/CubeShim/address and logs to the same directory. Process-level resource limits (such as RLIMIT_NOFILE) are configured in set_process ([main.rs L91-L140]) to prevent resource exhaustion during high-density container deployments.

Can CubeShim work with standard Kubernetes CRI implementations?

Yes. Because CubeShim implements the standard containerd Shim v2 API, it requires no modifications to the Kubernetes kubelet. Users simply specify the cube runtime class in Pod specifications, and containerd manages the Micro-VM through the standard CRI CreateContainer and RunPodSandbox flows.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}