How to Deploy CubeSandbox Using Terraform on Cloud VMs or Bare Metal

Deploy CubeSandbox by provisioning Tencent Cloud infrastructure with Terraform, then bootstrap the CubeMaster and Cubelet containers via cloud-init scripts that install Docker, mount persistent storage at /var/lib/cubesandbox, and launch services using the configuration files from configs/single-node/*.yaml.

CubeSandbox is a lightweight container sandbox developed by Tencent Cloud that provides isolated execution environments through its tightly-coupled component architecture. The TencentCloud/CubeSandbox repository recommends Terraform infrastructure-as-code to automate the deployment of cloud VMs or bare-metal servers, ensuring reproducible environments across regions and enabling version-controlled infrastructure rollbacks.

Understanding the CubeSandbox Architecture

CubeSandbox consists of five tightly-coupled components packaged as Docker images. Understanding these roles is essential for proper Terraform configuration:

Component	Role
CubeMaster	Central controller that schedules containers, manages templates, and stores persistent data.
Cubelet	Runs on each compute node, creates and manages isolated containers (the "cubes").
Network-Agent	Provides a programmable overlay network, DNS, and egress proxy for the cubes.
CubeEgress	Optional reverse-proxy (NGINX + TPROXY) that enables outbound traffic control and TLS termination.
CubeLifecycle-Manager	Handles snapshot/clone/rollback lifecycle of templates.

All components run as Docker containers (or pre-built binaries) on VMs or bare-metal hosts, configured via YAML files located in the configs/ directory.

Terraform Deployment Workflow

The standard deployment flow provisions infrastructure then bootstraps software via cloud-init or remote-exec provisioners:

1. Provision the network – Create a VPC, subnet, routing table, and security group opening ports 22 (SSH), 80/443 (web UI), 2379 (external etcd if used), and custom overlay-network ports.
2. Create compute nodes – Deploy one CVM for the CubeMaster and additional CVMs for cubelets (or bare-metal servers). The user_data script installs Docker, pulls images, and starts services.
3. Attach storage – Mount data disks to /var/lib/cubesandbox on each node for template repositories, snapshots, and logs.
4. Configure the Network-Agent – Inject the VPC CIDR into network-agent.yaml to allocate IP ranges without conflict.
5. Optionally deploy CubeEgress – Run NGINX with TPROXY on a separate instance for outbound traffic control.

Configuring the Terraform Provider

Create a provider.tf file to configure the Tencent Cloud provider with version constraints:

terraform {
  required_version = ">= 1.5.0"
  required_providers {
    tencentcloud = {
      source  = "tencentcloudstack/tencentcloud"
      version = "~> 1.96"
    }
  }
}

provider "tencentcloud" {
  region = var.region
}

Define input variables in variables.tf:

variable "region" { default = "ap-guangzhou" }
variable "vpc_cidr" { default = "10.0.0.0/16" }
variable "subnet_cidr" { default = "10.0.1.0/24" }
variable "instance_type" { default = "S5.MEDIUM4" }
variable "ssh_key_name" { description = "Existing SSH key in Tencent Cloud" }

Defining Network Infrastructure

Create network.tf to provision the VPC, subnet, and security groups required by CubeSandbox:

resource "tencentcloud_vpc" "sandbox_vpc" {
  name       = "cubesandbox-vpc"
  cidr_block = var.vpc_cidr
}

resource "tencentcloud_subnet" "sandbox_subnet" {
  vpc_id            = tencentcloud_vpc.sandbox_vpc.id
  name              = "cubesandbox-subnet"
  cidr_block        = var.subnet_cidr
  availability_zone = "${var.region}1"
}

resource "tencentcloud_security_group" "sandbox_sg" {
  name        = "cubesandbox-sg"
  vpc_id      = tencentcloud_vpc.sandbox_vpc.id
  description = "Allow CubeSandbox traffic"
}

resource "tencentcloud_security_group_rule" "ssh" {
  security_group_id = tencentcloud_security_group.sandbox_sg.id
  protocol          = "tcp"
  port              = "22"
  cidr_ip           = "0.0.0.0/0"
  action            = "accept"
  direction         = "ingress"
}

/* Open ports required by CubeSandbox */
resource "tencentcloud_security_group_rule" "cube_ui" {
  security_group_id = tencentcloud_security_group.sandbox_sg.id
  protocol          = "tcp"
  port              = "80,443"
  cidr_ip           = "0.0.0.0/0"
  action            = "accept"
  direction         = "ingress"
}

Provisioning the Master Node

Create master_node.tf to deploy a single-node instance running both CubeMaster and Cubelet:

resource "tencentcloud_cvm" "master" {
  instance_name   = "cubesandbox-master"
  availability_zone = "${var.region}1"
  instance_type   = var.instance_type
  image_id        = "img-8to2c2ul"   # Ubuntu 22.04 LTS (replace with your preferred image)

  system_disk_type = "CLOUD_BASIC"
  system_disk_size = 50

  vpc_id     = tencentcloud_vpc.sandbox_vpc.id
  subnet_id  = tencentcloud_subnet.sandbox_subnet.id
  security_groups = [tencentcloud_security_group.sandbox_sg.id]

  key_name   = var.ssh_key_name

  # Attach a data disk for persistent storage

  data_disks {
    disk_type = "CLOUD_PREMIUM"
    disk_size = 100
  }

  # Cloud-init script – installs Docker and starts CubeMaster + Cubelet

  user_data = <<-EOF
    #!/bin/bash
    set -e
    apt-get update && apt-get install -y docker.io
    systemctl enable docker && systemctl start docker

    # Pull official CubeSandbox images

    docker pull tencentcloud/cubesandbox-master:latest
    docker pull tencentcloud/cubesandbox-cubelet:latest
    docker pull tencentcloud/cubesandbox-network-agent:latest

    # Create persistent directory

    mkdir -p /var/lib/cubesandbox
    mount /dev/vdb /var/lib/cubesandbox   # vdb is the data disk created above

    # Launch CubeMaster (uses the bundled config file)

    docker run -d \
      --name cubesandbox-master \
      -v /var/lib/cubesandbox:/var/lib/cubesandbox \
      -p 80:80 -p 443:443 \
      tencentcloud/cubesandbox-master:latest

    # Launch Cubelet (connects to the master automatically)

    docker run -d \
      --name cubesandbox-cubelet \
      -v /var/lib/cubesandbox:/var/lib/cubesandbox \
      --network host \
      tencentcloud/cubesandbox-cubelet:latest
  EOF
}

Add outputs.tf to capture connection details:

output "master_public_ip" {
  value = tencentcloud_cvm.master.public_ip
}

Deploying on Bare Metal

For bare-metal servers registered as CVM-B instances or managed via out-of-band BMC, use the same Terraform modules with the instance_type set to a bare-metal flavor (e.g., BM_HIGHCPU_X2). The user_data script executes on the host OS directly rather than through a hypervisor layer.

The TencentCloud/CubeSandbox repository provides a detailed guide at docs/zh/guide/bare-metal-deploy.md that covers BMC configuration, disk mounting, and Docker initialization specific to physical servers. This guide ensures proper hardware initialization before the CubeSandbox containers start.

Multi-Node Cluster Configuration

To scale beyond a single node, duplicate the tencentcloud_cvm resource for additional cubelets:

• Set instance_name to cubesandbox-node-<index>
• Mount the same network-agent.yaml configuration on each node
• Adjust node_ip_range in configs/single-node/cubelet.yaml to prevent IP conflicts
• The CubeMaster automatically discovers cubelets via internal VPC IPs

Each node requires the same persistent storage mount at /var/lib/cubesandbox and must have network access to the CubeMaster's API endpoints defined in configs/single-node/cubemaster.yaml.

Storage and Persistence Configuration

The CubeLifecycle-Manager stores template snapshots and logs on the attached data disk mounted at /var/lib/cubesandbox. In the Terraform configuration, the data_disks block provisions this storage, and the cloud-init script formats and mounts it before starting containers.

According to the repository's architecture, this path is configurable via the storage parameters in configs/single-node/cubemaster.yaml, allowing you to separate system disks from template repositories for performance and durability.

Summary

• CubeSandbox deploys as Docker containers across five components: CubeMaster, Cubelet, Network-Agent, CubeEgress, and CubeLifecycle-Manager.
• Terraform automates the full stack: VPC provisioning, security groups, CVM instances, and cloud-init bootstrapping.
• Cloud-init scripts install Docker, pull images from the Tencent Cloud registry, mount persistent storage at /var/lib/cubesandbox, and launch services using configs/single-node/*.yaml.
• Bare-metal deployments use the same Terraform patterns with CVM-B instance types and the guide at docs/zh/guide/bare-metal-deploy.md.
• Multi-node clusters require duplicated CVM resources with adjusted node_ip_range values in the cubelet configuration.

Frequently Asked Questions

What ports must be open in the security group for CubeSandbox?

CubeSandbox requires TCP 22 for SSH administration, TCP 80/443 for the web UI and API endpoints, and TCP 2379 if using an external etcd cluster. Additionally, the overlay network (managed by the Network-Agent) requires specific ports defined in network-agent/docs/ARCHITECTURE.md for inter-node communication.

How do I configure persistent storage for templates and snapshots?

Attach a data disk via the Terraform data_disks block and mount it to /var/lib/cubesandbox in your cloud-init script. This directory stores the template repository, lifecycle snapshots, and logs. The path is defined in configs/single-node/cubemaster.yaml and must be available before starting the CubeMaster container.

Can I deploy CubeSandbox on existing bare-metal servers without Tencent Cloud CVMs?

Yes. Register your servers as CVM-B instances or manage them via out-of-band BMC, then use the same Terraform configuration with a bare-metal instance_type (e.g., BM_HIGHCPU_X2). The repository provides specific instructions in docs/zh/guide/bare-metal-deploy.md for initializing the host OS, configuring storage, and starting the Docker containers outside of the standard hypervisor environment.

Where can I find example configurations for the CubeEgress proxy?

The CubeEgress component configuration is located at CubeEgress/nginx.conf in the repository. This file contains the NGINX and TPROXY settings required for outbound traffic control and TLS termination. You can inject this configuration via Terraform's file provisioner or include it in your cloud-init script when deploying the optional egress instance.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}