What Is Cubelet's Function in CubeSandbox: Node-Level Daemon Explained

Cubelet is the node-level daemon that translates high-level control plane requests into actual sandbox MicroVMs, handling lifecycle management, storage, networking, and health reporting on every compute node in a Cube Sandbox cluster.

Cubelet serves as the critical bridge between the control plane and hardware in the TencentCloud/CubeSandbox architecture. This node-level agent runs on every compute node to orchestrate MicroVM creation, manage persistent storage volumes, and maintain continuous communication with CubeMaster. Understanding Cubelet's function in CubeSandbox is essential for operators managing serverless container workloads built on KVM microvirtualization.

Core Responsibilities of Cubelet in CubeSandbox

MicroVM Lifecycle Management

Cubelet manages the complete lifecycle of sandbox MicroVMs using containerd's runc/v2 runtime. In Cubelet/cmd/cubelet/main.go, the daemon creates a containerd client, sets up the mount namespace, and handles creation, startup, stopping, cloning, and rollback operations for sandbox processes.

Storage and Volume Handling

The daemon prepares VM rootfs instances and manages volume snapshots through local XFS, reflink pools, and Cubecow snapshots. The implementation in Cubelet/storage/cubecow_volume_manager.go handles bind-mounts of host paths and coordinates snapshot operations persistently.

Network Provisioning

Cubelet obtains TAP file descriptors from the network-agent and configures virtual NICs for each MicroVM. The AllocateTap RPC in Cubelet/pkg/networkagentclient/client.go provides the interface necessary to establish network connectivity and enforce network policies.

Health and Status Reporting

Through Cubelet/services/server/server.go, Cubelet periodically transmits node metrics and sandbox status to CubeMaster. This reporting enables the scheduler to make informed placement decisions based on real-time node health and resource availability.

gRPC API for Control Plane Communication

Cubelet exposes RPC methods including CreateSandbox, DeleteSandbox, and ListSandbox that CubeMaster invokes to orchestrate cluster workloads. These service definitions reside in Cubelet/services/server/config/config.go alongside the generated protobuf implementations.

Namespace Isolation

To ensure filesystem isolation, Cubelet creates private mount namespaces for each node using functions needNewMnt and newCubeMnt in main.go. This mechanism re-enters the namespace when handling host-mounts, presenting a clean view of the host filesystem to each VM.

Key Implementation Files

The following source files define Cubelet's architecture according to the TencentCloud/CubeSandbox repository:

• Entry point: Cubelet/cmd/cubelet/main.go initializes the containerd client and mount namespace handling
• gRPC server: Cubelet/services/server/server.go implements the control plane interface and status reporting
• Storage engine: Cubelet/storage/cubecow_volume_manager.go and Cubelet/storage/cubecow_snapshot_artifacts.go manage volumes and snapshots
• Network client: Cubelet/pkg/networkagentclient/client.go handles TAP allocation and network configuration
• CLI client: Cubelet/cmd/cubecli/main.go provides the user-facing interface for sandbox operations

Practical Code Examples

Creating a Sandbox via CLI

The cubecli utility communicates with Cubelet's gRPC interface to create MicroVMs:

# Run on a compute node to create a new sandbox

./cubecli sandbox create \
    --template-id tpl-12345 \
    --metadata '{"host-mount":[{"hostPath":"/data/models","containerPath":"/mnt/models"}]}' \
    --network '{"cidr":"192.168.10.0/24"}'

This command invokes the CreateSandbox method defined in Cubelet/services/server/server.go, triggering Cubelet to prepare storage, configure networking, and start the microVM.

Cloning Volume Snapshots

Cubelet's storage API enables efficient snapshot cloning:

import "github.com/tencentcloud/CubeSandbox/Cubelet/storage"

func cloneSnapshot(srcID, dstID string) error {
    // Use the Cubecow snapshot engine to copy a template snapshot
    return storage.CloneSnapshot(srcID, dstID)
}

The CloneSnapshot function is implemented in Cubelet/storage/cubecow_snapshot_artifacts.go.

Allocating Network Interfaces

Access the network-agent to obtain TAP file descriptors:

import "github.com/tencentcloud/CubeSandbox/Cubelet/pkg/networkagentclient"

func getTapFD() (int, error) {
    client, _ := networkagentclient.NewClient()
    return client.AllocateTap()
}

Summary

• Cubelet functions as the node-level daemon in CubeSandbox, translating CubeMaster requests into concrete MicroVMs
• It manages the complete MicroVM lifecycle using containerd's runc/v2 runtime through Cubelet/cmd/cubelet/main.go
• Storage operations including XFS volumes and Cubecow snapshots are handled via cubecow_volume_manager.go
• Network provisioning occurs through TAP file descriptor allocation in networkagentclient/client.go
• Status reporting keeps CubeMaster informed of node health via services/server/server.go
• Namespace isolation ensures filesystem separation using needNewMnt and newCubeMnt functions

Frequently Asked Questions

What is the relationship between Cubelet and CubeMaster?

CubeMaster acts as the control plane that makes scheduling decisions, while Cubelet runs on each compute node to execute those decisions. Cubelet exposes a gRPC API that CubeMaster calls to create, delete, and monitor sandboxes, then reports back node metrics and MicroVM status to inform scheduling.

How does Cubelet handle storage snapshots?

Cubelet implements the Cubecow snapshot engine in Cubelet/storage/cubecow_volume_manager.go and cubecow_snapshot_artifacts.go. It supports local XFS with reflink pools and provides the CloneSnapshot function to efficiently copy template snapshots without duplicating data blocks.

What runtime does Cubelet use for MicroVMs?

Cubelet uses containerd's runc/v2 runtime to manage sandbox processes. The integration is initialized in Cubelet/cmd/cubelet/main.go, where Cubelet creates a containerd client and configures the mount namespace before starting MicroVM processes.

How does Cubelet isolate sandbox filesystems?

Cubelet creates private mount namespaces for each node using the needNewMnt and newCubeMnt functions in main.go. When handling host-mounts, it re-enters these namespaces to ensure each MicroVM sees a clean, isolated view of the host filesystem.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}