How to Integrate CubeSandbox with Existing Container Orchestration Systems: A Kubernetes CRI Guide

Integrate CubeSandbox by deploying Cubelet as a CRI-compatible runtime plugin that replaces the standard container runtime, allowing Kubernetes to schedule hardware-isolated micro-VMs instead of traditional containers.

TencentCloud/CubeSandbox provides a set of tightly-coupled components designed to integrate with existing container orchestration platforms like Kubernetes. The core integration strategy centers on Cubelet, a CRI-compatible shim that registers itself with the kubelet to schedule and run hardware-isolated micro-VM sandboxes. This approach enables you to leverage existing Kubernetes infrastructure while gaining the security and isolation benefits of CubeSandbox's lightweight virtualization.

CubeSandbox Architecture for Container Orchestration

The integration relies on six core components that replace or augment standard Kubernetes infrastructure:

• CubeAPI – A RESTful control-plane gateway that exposes sandbox creation, execution, and termination APIs compatible with the E2B specification.
• CubeMaster – The cluster manager that receives API calls, validates templates, and forwards sandbox lifecycle requests to appropriate Cubelet instances.
• Cubelet – A CRI runtime plugin that registers with the kubelet via the standard container runtime interface. It accepts pod creation requests and spins up Cube Hypervisor micro-VMs for each sandbox.
• CubeVS – An eBPF-based virtual switch providing kernel-level network isolation and egress policy enforcement.
• CubeEgress – An OpenResty proxy implementing L7 domain filtering, credential injection, and audit logging for outbound traffic.
• CubeShim – Implements the containerd Shim v2 API, allowing Cube Hypervisor to appear as a standard container runtime to the kubelet.

In Cubelet/plugins/cube/runtime/plugin.go, the CRI plugin implementation handles the translation between Kubernetes pod specifications and CubeSandbox micro-VM provisioning.

Integration Steps for Kubernetes

Register Cubelet as a CRI Runtime

Cubelet ships with a CRI plugin located at plugins/cube/runtime/plugin.go that the kubelet discovers via the standard --container-runtime-endpoint flag. This makes Cubelet appear as a drop-in replacement for Docker or containerd.

Configure the kubelet on each node to point to the Cubelet socket:

# /etc/kubernetes/kubelet.conf or systemd service flags

--container-runtime=remote
--container-runtime-endpoint=unix:///run/cubelet/cubelet.sock
--runtime-request-timeout=2m

Configure Control-Plane Endpoints

Set the environment variable CUBE_API_URL (or CUBE_PROXY_NODE_IP for data-plane traffic) so Cubelet can reach the CubeAPI service. According to docs/guide/connect-existing-cluster.md, these variables also enable "IP-direct dialing" which eliminates the need for wildcard DNS configuration.

In Cubelet/pkg/config/config.go, the runtime configuration handles these API endpoints and proxy settings.

Deploy the Data-Plane (CubeProxy)

The kubelet on each node contacts CubeProxy for sandbox traffic. When using Cube native SDKs, set CUBE_PROXY_NODE_IP and CUBE_PROXY_PORT_HTTP to bypass DNS completely (Option A). For SPA front-ends or production clusters, use wildcard DNS (Option C) or a dev-sidecar (Option D).

The Cubelet/services/server/server.go file contains the HTTP server implementation that receives sandbox lifecycle requests from CubeMaster and coordinates with CubeProxy.

Template Distribution Across Nodes

CubeMaster distributes OCI-based templates to every Cubelet in the cluster. Templates are stored once per node and automatically replicated, ensuring new sandbox instances start instantly.

The template handling logic resides in CubeMaster/pkg/templatecenter/template_image.go, which propagates images to Cubelet nodes. The CubeProxy/sidecar/internal/registry/registry.go file manages the registry logic that Cubelet uses to pull images for sandbox creation.

Network and Security Configuration

CubeVS provides per-sandbox network namespaces at the kernel level, while CubeEgress enforces L7 egress policies. Both components are automatically wired into the sandbox lifecycle by Cubelet, as documented in docs/architecture/network.md.

Deployment Examples

Deploy CubeAPI as a control-plane Deployment:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: cube-api
spec:
  selector:
    matchLabels:
      app: cube-api
  template:
    metadata:
      labels:
        app: cube-api
    spec:
      containers:
        - name: cube-api
          image: tencentcloud/cube-api:latest
          env:
            - name: CUBE_PROXY_NODE_IP
              valueFrom:
                fieldRef:
                  fieldPath: status.podIP
            - name: CUBE_API_URL
              value: "http://$(CUBE_PROXY_NODE_IP):3000"
          ports:
            - containerPort: 3000

Install Cubelet as a DaemonSet on each node:

apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: cubelet
spec:
  selector:
    matchLabels:
      app: cubelet
  template:
    metadata:
      labels:
        app: cubelet
    spec:
      hostNetwork: true
      containers:
        - name: cubelet
          image: tencentcloud/cubelet:latest
          args:
            - "--container-runtime-endpoint"
            - "unix:///run/cubelet/cubelet.sock"
          securityContext:
            privileged: true
          volumeMounts:
            - name: kubelet-socket
              mountPath: /run/cubelet
      volumes:
        - name: kubelet-socket
          hostPath:
            path: /run/cubelet
            type: DirectoryOrCreate

Create a pod that runs inside a Cube sandbox by specifying the runtimeClassName:

apiVersion: v1
kind: Pod
metadata:
  name: sandbox-demo
spec:
  runtimeClassName: cubelet
  containers:
    - name: app
      image: python:3.11-slim
      command: ["python", "-c", "print('Hello from a Cube sandbox!')"]

Summary

• Cubelet acts as a CRI plugin in Cubelet/plugins/cube/runtime/plugin.go, enabling drop-in replacement of containerd or Docker without modifying pod specifications beyond runtimeClassName.
• Environment variables CUBE_API_URL and CUBE_PROXY_NODE_IP configure control-plane and data-plane connectivity, supporting IP-direct dialing to avoid DNS dependencies.
• OCI-based templates distribute automatically across the cluster via CubeMaster, with image handling logic in CubeMaster/pkg/templatecenter/template_image.go.
• Network isolation combines CubeVS for kernel-level namespaces and CubeEgress for L7 filtering, both managed automatically by Cubelet.
• Privileged access is required for Cubelet to access KVM hardware acceleration on cluster nodes.

Frequently Asked Questions

What is Cubelet's role in the integration?

Cubelet implements the standard CRI and containerd Shim v2 interfaces, allowing the Kubernetes kubelet to treat sandboxes exactly like containers. Located in Cubelet/plugins/cube/runtime/plugin.go, it registers itself as a remote runtime endpoint and translates pod creation requests into micro-VM provisioning commands for the Cube Hypervisor.

How does CubeSandbox handle networking compared to standard containers?

CubeSandbox replaces standard container networking with CubeVS, an eBPF-based virtual switch that provides kernel-level network isolation, and CubeEgress, an OpenResty proxy for L7 domain filtering. These components automatically wire into each sandbox lifecycle, enforcing egress policies and audit logging without requiring manual CNI configuration.

Can I use CubeSandbox with orchestrators other than Kubernetes?

Yes. Because Cubelet implements the standard Container Runtime Interface (CRI), it works with any CRI-compatible orchestration system, including OpenShift, TKE, and other Kubernetes distributions. The architecture is scheduler-agnostic, requiring only that the orchestrator supports remote container runtime endpoints.

What permissions does Cubelet require on cluster nodes?

Cubelet requires privileged security context access to utilize KVM hardware acceleration for micro-VM creation. The DaemonSet configuration must run with hostNetwork: true and privileged container permissions to access the underlying hypervisor capabilities and manage network namespaces at the kernel level.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}