# How to Perform Multi-Node Cluster Deployment with the CubeMaster Orchestrator in CubeSandbox

> Learn to deploy a multi-node CubeSandbox cluster using the CubeMaster orchestrator. Configure compute nodes and install CubeMaster for seamless cluster management. Get started today.

- Repository: [Tencent Cloud/CubeSandbox](https://github.com/TencentCloud/CubeSandbox)
- Tags: how-to-guide
- Published: 2026-07-03

---

**Deploy a multi-node CubeSandbox cluster by configuring compute nodes with `ONE_CLICK_DEPLOY_ROLE=compute`, pointing them to the CubeMaster control plane, and executing [`install-compute.sh`](https://github.com/TencentCloud/CubeSandbox/blob/main/install-compute.sh) to register via the `/internal/meta` API.**

CubeSandbox separates the **control plane** (CubeMaster, CubeAPI, Redis, MySQL) from the **data plane** (Cubelet, CubeShim, CubeHypervisor) to enable horizontal scaling. This guide covers expanding a single control node into a production-ready multi-node cluster by adding compute-only nodes that automatically register with the CubeMaster orchestrator. All procedures reference the deployment automation in the TencentCloud/CubeSandbox repository.

## Architecture Overview

CubeSandbox clusters consist of one control node and multiple compute nodes. The control node hosts CubeMaster, which exposes an internal registration API at `/internal/meta` on port **8089**. Compute nodes run a minimal stack—**Cubelet**, **network-agent**, and **CubeShim**—and register themselves as available capacity to the orchestrator.

```

Control Node (CubeMaster:8089)
         │
         │ /internal/meta API
         ▼
    ┌────────┬────────┐
    ▼        ▼        ▼
Compute1  Compute2  ComputeN

```

The control node runs the full stack while also acting as a compute resource. Additional compute nodes run only the sandbox runtime components and report health status to CubeMaster.

## Prerequisites

Before adding compute nodes, ensure your infrastructure meets these requirements:

- **Hardware**: Physical or bare-metal servers; nested virtualization is **not** supported
- **CPU Architecture**: `x86_64` or `aarch64` with KVM enabled (`ls /dev/kvm` must return the device)
- **Operating System**: Linux with Docker installed and running
- **Network**: Compute nodes must reach the control node's CubeMaster port **8089**
- **Control Node**: Already deployed via the Self-Build Deployment guide with CubeMaster running

## Step-by-Step Multi-Node Deployment

### Prepare the Release Bundle

Build the one-click deployment bundle on your control node, then distribute it to each compute node:

```bash

# On control node

make one-click-bundle VERSION=0.4.0
scp cube-sandbox-one-click-0.4.0.tar.gz user@compute-node:/tmp/

# On compute node

tar -xzf cube-sandbox-one-click-0.4.0.tar.gz
cd cube-sandbox-one-click-0.4.0

```

### Configure Environment Variables

Copy the example environment file and set three critical variables to define the node's role and control plane endpoint:

```bash
cp env.example .env

```

Edit `.env` with these values:

```text
ONE_CLICK_DEPLOY_ROLE=compute
CUBE_SANDBOX_NODE_IP=<current-node-ip>
ONE_CLICK_CONTROL_PLANE_IP=<control-node-ip>

```

- **ONE_CLICK_DEPLOY_ROLE**: Must be set to `compute` to install only data-plane components
- **CUBE_SANDBOX_NODE_IP**: Primary NIC IP address of the current host (not `127.0.0.1`)
- **ONE_CLICK_CONTROL_PLANE_IP**: IP address of the control node; the installer automatically appends port `:8089`

If CubeMaster uses a non-standard port, override with `ONE_CLICK_CONTROL_PLANE_CUBEMASTER_ADDR=<ip>:<port>` instead of the `_IP` variable.

### Execute the Compute Node Installer

Run the compute-specific installer script located at [`deploy/one-click/install-compute.sh`](https://github.com/TencentCloud/CubeSandbox/blob/main/deploy/one-click/install-compute.sh):

```bash
sudo ./install-compute.sh

```

This script performs the following actions:

1. Installs **Cubelet**, **network-agent**, **CubeShim**, **CubeImage**, and **Cube-Kernel-SCF**
2. Configures Cubelet's `meta_server_endpoint` to point to the control plane's CubeMaster address
3. Starts only the host processes `network-agent` and `cubelet`
4. Registers the node via the `/internal/meta` API and begins reporting health metrics

The script automatically stops any existing deployment if re-run, making it safe for updates.

### Verify Node Registration

Validate the deployment from both perspectives:

**On the compute node:**

```bash
sudo ./smoke.sh

```

**On the control node:**

```bash
curl http://<control-node-ip>:8089/internal/meta/nodes

```

The API response should list the new compute node's IP address with a `healthy` status, confirming successful registration with the CubeMaster orchestrator.

### Optimize CubeMaster Scheduling (Optional)

For clusters larger than a single node, enable weighted scoring in [`cubemaster.yaml`](https://github.com/TencentCloud/CubeSandbox/blob/main/cubemaster.yaml) to distribute workloads across the top-rated nodes rather than always selecting the first available:

```yaml
scheduler:
  priority_select_num: 3
  score:
    enable_scorers:
      - real_time_weighted_average
    resource_weights:
      mvm_num: 2
      local_create_num: 3
      quota_cpu_usage: 1
      quota_mem_usage: 1
    plugin_conf:
      real_time_weighted_average:
        weight: 1.0
        enable_weight_factors:
          - mvm_num
          - local_create_num
          - quota_cpu_usage
          - quota_mem_usage

```

Restart CubeMaster after applying these changes to activate the multi-node scheduling logic.

## Operational Commands and Log Locations

Manage compute nodes using these utilities:

- **Stop services**: `sudo ./down.sh` (stops only Cubelet and network-agent)
- **Re-install**: Re-run `sudo ./install-compute.sh` (auto-stops existing processes)

Log file locations for troubleshooting:

| Component | Path |
|-----------|------|
| Cubelet | `/data/log/Cubelet/` |
| CubeShim | `/data/log/CubeShim/` |
| CubeVmm | `/data/log/CubeVmm/` |
| PID files | `/var/run/cube-sandbox-one-click/` |
| Stdout/Stderr | `/var/log/cube-sandbox-one-click/` |

## Troubleshooting Multi-Node Registration

**Node fails to appear in CubeMaster:**

Verify the compute node can reach the control plane on port 8089. Check firewall rules and confirm `.env` contains the correct `ONE_CLICK_CONTROL_PLANE_IP` or `ONE_CLICK_CONTROL_PLANE_CUBEMASTER_ADDR`.

**Registration errors in logs:**

Examine `/data/log/Cubelet/` for connection failures. Ensure `CUBE_SANDBOX_NODE_IP` is set to a routable address, not localhost, and that the CubeMaster service is active on the control node.

## Summary

- CubeSandbox clusters separate control plane services from data plane compute nodes, with CubeMaster orchestrating via the `/internal/meta` API on port 8089
- Compute nodes require `ONE_CLICK_DEPLOY_ROLE=compute` and proper `ONE_CLICK_CONTROL_PLANE_IP` configuration to register correctly
- The [`install-compute.sh`](https://github.com/TencentCloud/CubeSandbox/blob/main/install-compute.sh) script automates installation, configuration, and registration of compute nodes with the CubeMaster orchestrator
- Verify deployments using [`./smoke.sh`](https://github.com/TencentCloud/CubeSandbox/blob/main/./smoke.sh) locally and `curl /internal/meta/nodes` from the control node
- Scale scheduling efficiently by enabling weighted scoring with `priority_select_num` in [`cubemaster.yaml`](https://github.com/TencentCloud/CubeSandbox/blob/main/cubemaster.yaml)

## Frequently Asked Questions

### What distinguishes a control node from a compute node in CubeSandbox?

A control node runs the complete stack including CubeMaster, CubeAPI, Redis, MySQL, and CoreDNS, while also functioning as a compute resource. A compute node runs only the sandbox runtime components—Cubelet, network-agent, and CubeShim—and registers its capacity with the control node's CubeMaster orchestrator.

### Which network port must be accessible between compute nodes and the control plane?

Compute nodes must reach the control node's CubeMaster service on **port 8089** to access the `/internal/meta` registration and health-checking API. Ensure firewalls allow outbound connections from compute nodes to this port on the control node.

### How can I confirm that a compute node successfully registered with CubeMaster?

On the compute node, run `sudo ./smoke.sh` to verify local service health. On the control node, execute `curl http://127.0.0.1:8089/internal/meta/nodes` and verify the JSON response contains the compute node's IP address with a `healthy` status indicator.

### Does CubeSandbox support nested virtualization for compute nodes?

No. CubeSandbox requires physical or bare-metal servers with KVM enabled. Nested virtualization environments will fail to provide the hardware acceleration required by the CubeHypervisor component.