Strategies for High-Density Deployment of Thousands of CubeSandbox Instances

High-density deployment of thousands of CubeSandbox instances relies on RustVMM-based MicroVMs, Copy-on-Write storage, kernel page sharing, and pre-allocated TAP device pools to achieve approximately 25 MiB of memory overhead per sandbox while supporting over 1,000 instances on a single host.

CubeSandbox, developed by TencentCloud, is engineered to run thousands of isolated sandbox instances on a single bare-metal node. Achieving this level of density requires specific architectural choices and runtime configurations that minimize per-instance overhead. This guide explains the strategies for high-density deployment of thousands of sandbox instances based on the actual source code and benchmark data from the CubeSandbox repository.

Architectural Foundations for Extreme Density

CubeSandbox achieves extreme density through a combination of virtualization technologies and resource-sharing mechanisms. Each component is designed to reduce the marginal cost of running an additional sandbox.

MicroVM Isolation with RustVMM

Each sandbox runs in a lightweight MicroVM built on RustVMM and KVM. Unlike traditional virtual machines, these MicroVMs provide hardware-level isolation without the heavy overhead of a full operating system. This architecture is defined in docs/architecture/overview.md and enables the rapid instantiation required for high-density workloads.

Copy-on-Write Storage

Sandbox filesystems utilize XFS reflink volumes with Copy-on-Write (CoW) semantics. As documented in the performance benchmark, this approach materializes only changed blocks, keeping the disk footprint minimal and enabling fast snapshot and clone operations. When deploying thousands of instances from a common template, the storage overhead remains nearly constant regardless of instance count.

Kernel Page Sharing

The host kernel shares identical pages among all MicroVMs. According to the benchmark report in docs/blog/posts/2026-06-01-cubesandbox-perf-benchmark.md, this mechanism reduces the memory footprint of idle sandboxes to approximately 25 MiB each. On a 375 GiB bare-metal node (Tencent Cloud BMI5), tests demonstrate a linear increase from ~21 MiB (100 sandboxes) to ~26 MiB (1000 sandboxes), allowing a single node to host over 1,000 idle sandboxes while leaving capacity for active workloads.

Soft-Dirty Page Tracking

Snapshots record only pages dirtied since the last checkpoint using soft-dirty page tracking. While this feature is noted as landing in a future release, the benchmark report indicates it will reduce pause latency from ~550 ms to ~60 ms, significantly improving throughput during auto-pause operations.

Pre-Allocated TAP Device Pools

The network-agent service maintains a pre-created pool of virtual NICs to eliminate per-sandbox TAP creation latency. The implementation in network-agent/internal/service/tap_lifecycle.go includes an InitPool function that creates the target number of devices at startup:

func (t *TapLifecycle) InitPool(target int) error {
    // Called at startup; creates `target` TAP devices.
    for i := 0; i < target; i++ {
        if err := t.createTap(); err != nil {
            return err
        }
    }
    return nil
}

Stateless Control Plane

CubeMaster, CubeAPI, and CubeProxy hold no sandbox state; all coordination persists in Redis. This stateless design, documented in docs/architecture/overview.md, allows horizontal scaling of the scheduler without adding per-sandbox state, making it possible to distribute thousands of instances across multiple nodes seamlessly.

Runtime Configuration for Maximum Density

Specific configuration parameters in Cubelet/config/config.toml and CubeMaster/config/cubemaster.yaml must be tuned to achieve optimal density.

TAP Pool Sizing

Set tap_init_num in the Cubelet configuration to match or exceed your target sandbox count. For 1,000 sandboxes, configure:

[plugins."io.cubelet.internal.v1.network"]
tap_init_num = 1000

After modifying this value, restart the network-agent service to apply the change:

systemctl restart cube-sandbox-network-agent.service

Resource Release Settings

Configure host.quota.paused_resource_release_ratio to 1.0 in cubemaster.yaml to release all CPU and memory resources when a sandbox is paused:

host:
  quota:
    paused_resource_release_ratio: 1.0

This setting allows the scheduler to treat paused sandboxes as consuming zero resources, maximizing available capacity for new instances.

Scheduler Tuning

Adjust the scheduler configuration to prevent hotspotting and optimize distribution:

scheduler:
  priority_select_num: 3
  score:
    enable_scorers:
      - real_time_weighted_average
    resource_weights:
      mvm_num: 2
      local_create_num: 3
      quota_cpu_usage: 1
      quota_mem_usage: 1
    plugin_conf:
      real_time_weighted_average:
        weight: 1.0
        enable_weight_factors:
          - mvm_num
          - local_create_num
          - quota_cpu_usage
          - quota_mem_usage

Setting priority_select_num to 3 or higher allows the scheduler to select among the top-scored nodes rather than always picking the highest-scored node, preventing resource contention on individual hosts.

Operational Strategies

Beyond architecture and configuration, specific operational practices maximize density and throughput.

Auto-Pause and Auto-Resume

Idle sandboxes are automatically paused via the POST /sandboxes/:id/pause endpoint and resumed on demand. When combined with paused_resource_release_ratio: 1.0, this automatically frees resources for new instances, effectively increasing the total number of sandboxes that can be managed by a single host over time.

Batch Creation and Concurrency Tuning

The cube-bench tool demonstrates that 20-concurrent creation yields the optimal latency-throughput trade-off on a 96-core host. The benchmark data shows:

• 1 concurrent: 55 ms per sandbox
• 10 concurrent: 9.9 ms per sandbox
• 20 concurrent: 5.5 ms per sandbox (peak throughput)
• 50 concurrent: 6.8 ms per sandbox (slightly higher due to queue depth)

Run the benchmark with:

cd examples/cube-bench
make
export E2B_API_URL=http://<control-node-ip>:3000
export E2B_API_KEY=e2b_000000
export CUBE_TEMPLATE_ID=<template-id>

./bin/cube-bench -c 20 -n 300 -w 3 -m create-only

Snapshot-Based Workflows

Snapshot, clone, and rollback operations are near-instant (≈50 ms for snapshot, ≤10 ms per-instance when run concurrently). Using snapshots to branch workloads avoids the overhead of recreating sandboxes from templates. The benchmark scripts in examples/snapshot-rollback-clone/bench_pause_resume_concurrency.py demonstrate these operations at high concurrency.

Multi-Node Scaling

When a single node reaches CPU or memory limits, add compute-only nodes that run only Cubelet and network-agent. The stateless control plane automatically balances scheduling across nodes via the configurable scheduler, as documented in docs/guide/multi-node-deploy.md.

Implementation Examples

Python Bulk Creation with Auto-Pause

Use the Python SDK to create and pause sandboxes in batches:

import os
from e2b_code_interpreter import Sandbox

os.environ["E2B_API_URL"] = "http://<control-node-ip>:3000"
os.environ["E2B_API_KEY"] = "e2b_000000"
os.environ["CUBE_TEMPLATE_ID"] = "<template-id>"

def create_many(n):
    sandboxes = []
    for _ in range(n):
        sb = Sandbox.create(template=os.environ["CUBE_TEMPLATE_ID"])
        sandboxes.append(sb)
    return sandboxes

# Create 1000 sandboxes

sandbox_list = create_many(1000)

# Pause all to free resources

for sb in sandbox_list:
    sb.pause()

Configuring High-Density Parameters

Apply all configuration changes for a 1,000-sandbox deployment:

# 1. Configure Cubelet TAP pool

vi /usr/local/services/cubetoolbox/Cubelet/config/config.toml

# Add:

[plugins."io.cubelet.internal.v1.network"]
tap_init_num = 1000

# 2. Restart network agent

systemctl restart cube-sandbox-network-agent.service

# 3. Configure CubeMaster scheduler and resource release

vi /usr/local/services/cubetoolbox/CubeMaster/config/cubemaster.yaml

# Add:

scheduler:
  priority_select_num: 3
host:
  quota:
    paused_resource_release_ratio: 1.0

Summary

• MicroVM architecture based on RustVMM provides hardware isolation with minimal overhead.
• Copy-on-Write storage and kernel page sharing keep per-instance memory overhead to approximately 25 MiB.
• Pre-allocate TAP devices by setting tap_init_num to your target sandbox count before starting the network-agent.
• Set paused_resource_release_ratio to 1.0 to free all resources from paused sandboxes, maximizing scheduling capacity.
• Use 20-concurrent creation for optimal throughput, yielding ~5.5 ms amortized latency per sandbox.
• Leverage snapshots for sub-100 ms branching operations instead of recreating instances from templates.
• Scale horizontally with compute-only nodes when single-host limits are reached; the stateless control plane handles distribution automatically.

Frequently Asked Questions

What is the memory overhead per CubeSandbox instance?

Each sandbox consumes approximately 25 MiB of additional memory on the host after Copy-on-Write and kernel page sharing optimizations. Benchmarks on a 375 GiB bare-metal node show this overhead remains stable from 100 to 1,000 instances, enabling high-density deployment of thousands of sandbox instances on a single server.

How do I prevent TAP device creation from becoming a bottleneck?

Configure tap_init_num in Cubelet/config/config.toml to pre-create a pool of virtual NICs equal to or greater than your maximum sandbox count. For example, set tap_init_num = 1000 to support 1,000 sandboxes, then restart the network-agent service. This eliminates runtime TAP creation latency, which would otherwise impede high-density deployment.

What concurrency level should I use for batch creation?

Use 20 concurrent operations for optimal throughput. According to the cube-bench tool in examples/cube-bench, this concurrency level achieves the best amortized latency of approximately 5.5 ms per sandbox on a 96-core host, compared to 55 ms at single concurrency or degraded performance at 50+ concurrency due to queue depth.

How does auto-pause help achieve higher density?

When auto-pause is enabled and paused_resource_release_ratio is set to 1.0, paused sandboxes release all CPU and memory resources to the scheduler. This allows the system to treat idle instances as consuming zero resources, effectively increasing the total number of sandboxes that can be managed across the cluster over time.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}