CubeSandbox Performance Characteristics for AI Agent Workloads: Latency, Density, and Throughput Benchmarks

CubeSandbox achieves sub-50ms cold-start latency, scales to 180 sandbox creations per second, and maintains approximately 25 MiB memory overhead per instance through KVM MicroVMs and copy-on-write storage, delivering the ultra-low latency and high concurrency required for AI agent workloads.

TencentCloud/CubeSandbox is engineered specifically for AI agent workloads that demand millisecond-scale responsiveness and massive horizontal scaling. The architecture combines a stateless control plane with a KVM MicroVM data plane, leveraging copy-on-write (CoW) storage and eBPF networking to minimize per-instance resource consumption. Understanding these performance characteristics is essential for architects designing large-scale LLM-driven services.

Cold-Start Latency and Concurrent Throughput

The benchmark reports in docs/blog/posts/2026-06-01-cubesandbox-perf-benchmark.md demonstrate that CubeSandbox optimizes for both single-instance latency and bulk creation throughput.

Single Sandbox Creation Performance

Creating a single sandbox from a template averages ~48 ms, with a minimum of 43.5 ms and a p95 of 57.4 ms. This metric represents the time from API call to a fully running sandbox ready to accept workloads, measured using the core operations benchmark suite.

High-Concurrency Scaling

Under concurrent load, throughput rises to approximately 180 sandboxes per second when creating 20 sandboxes simultaneously. At this concurrency level, the per-sandbox amortized latency drops to roughly 5.5 ms, demonstrating that CubeSandbox's control plane efficiently batches operations and reduces per-instance overhead through parallelization.

Memory Density and Per-Instance Overhead

CubeSandbox achieves high single-host density through XFS reflink copy-on-write (CoW) storage, implemented in the CubeCoW engine as documented in cubecow/README.md.

For a standard 2 vCPU / 2 GiB sandbox configuration, the per-instance memory overhead converges to approximately 25 MiB. This efficiency enables deployments of 1,000 sandboxes consuming only ~25 GiB on a 375 GiB host machine, leaving the majority of RAM available for AI agent computation. The cubelet/config/config.toml file exposes TAP pool sizing and other tunables that directly impact this density.

Snapshot, Clone, and Lifecycle Performance

AI agent workloads frequently require checkpointing and restoration. CubeSandbox provides O(1) snapshot operations through CoW semantics while maintaining predictable latency for dirty-page tracking.

Snapshot Creation Latency

Snapshot creation time scales linearly with dirty-page size:

• ~47 ms for a clean sandbox with minimal dirty pages
• ~40 ms additional per 100 MiB of dirty data
• Up to ~487 ms for sandboxes with 1 GiB of dirty memory

This behavior is benchmarked in examples/snapshot-rollback-clone/bench_snapshot_concurrency.py.

Create-from-Snapshot and Clone Operations

Restoring from a snapshot or cloning exhibits constant-time characteristics regardless of snapshot size:

• Create-from-snapshot: 60–85 ms (uses CoW rather than full data copy)
• Single clone: ~220 ms
• Amortized clone (100 clones at 50 concurrent): ~5.4 ms per clone

These measurements are available in examples/snapshot-rollback-clone/bench_clone_concurrency.py.

Pause, Resume, and Rollback Latency

Lifecycle operations show strong concurrency amortization:

• Resume: ~42 ms single, ~3.6 ms per sandbox at 10 concurrent
• Rollback: ~82 ms single (full-memory copy mode), ~27 ms at 10 concurrent
• Pause: ~558 ms single (full-copy mode), ~68 ms at 10 concurrent; future soft-dirty mode will reduce single-pause latency to ~60 ms

Networking and Storage Architecture

CubeSandbox employs eBPF-based CubeVS networking to provide line-speed NAT/DNAT without iptables rules, as detailed in docs/zh/architecture/overview.md. This kernel-level enforcement eliminates the per-packet overhead of traditional bridge networking, critical for AI agents making high-frequency API calls. The CubeShim component (see CubeShim/README.md) bridges containerd operations to the MicroVM layer while maintaining these performance characteristics.

Measuring Performance Programmatically

You can reproduce these benchmarks using the official SDK and CLI tools.

Measure cold-start latency using the Python SDK:

import os, time
from e2b_code_interpreter import Sandbox

template_id = os.getenv("CUBE_TEMPLATE_ID")

def measure_startup():
    start = time.perf_counter()
    with Sandbox.create(template=template_id) as sandbox:
        end = time.perf_counter()
        print(f"Cold‑start latency: {(end-start)*1000:.1f} ms")
        print(sandbox.run_code("print('ready')").output)

measure_startup()

Benchmark snapshot operations with dirty-page tracking:

import os, time
from e2b_code_interpreter import Sandbox

template_id = os.getenv("CUBE_TEMPLATE_ID")

with Sandbox.create(template=template_id) as sb:
    sb.run_code("a = [i for i in range(10**6)]; print(len(a))")
    t0 = time.perf_counter()
    snap = sb.create_snapshot()
    t1 = time.perf_counter()
    print(f"Snapshot creation: {(t1-t0)*1000:.1f} ms")
    t2 = time.perf_counter()
    sb.rollback(snap.id)
    t3 = time.perf_counter()
    print(f"Rollback latency: {(t3-t2)*1000:.1f} ms")

Execute high-concurrency creation tests using the Go benchmark tool:

cd examples/cube-bench && make
export E2B_API_URL=http://<host>:3000
export CUBE_TEMPLATE_ID=<tpl-id>
./bin/cube-bench -c 20 -n 300 -w 3 -m create-only -o report.json

Configuration Tuning for AI Workloads

Optimizing CubeSandbox for AI agents requires adjusting the TAP pool size and memory limits in cubelet/config/config.toml. The default configuration balances density with isolation, but workloads with heavy LLM inference may require adjusted dirty-page tracking thresholds to minimize snapshot latency. The CubeShim implementation ensures these configurations propagate correctly to the underlying KVM MicroVMs via the containerd shim API.

Summary

• CubeSandbox achieves ~48 ms average cold-start latency for AI agents, with p95 under 60 ms.
• At 20 concurrent creations, throughput reaches 180 sandboxes per second with per-instance latency dropping to ~5.5 ms.
• XFS reflink CoW reduces memory overhead to ~25 MiB per 2 GiB sandbox, enabling 1,000 instances on modest hardware.
• Snapshot creation scales linearly with dirty data (+40 ms per 100 MiB), while restore operations remain constant at 60–85 ms.
• eBPF networking (CubeVS) and kernel-level isolation provide line-speed throughput without iptables overhead.

Frequently Asked Questions

What is the cold-start latency of CubeSandbox for a single AI agent?

CubeSandbox achieves an average cold-start latency of ~48 ms when creating a single sandbox from a template, with a minimum of 43.5 ms and a 95th percentile of 57.4 ms. This measurement covers the complete initialization from API request to runnable state, as documented in the core operations benchmark report.

How does CubeSandbox handle memory overhead for thousands of concurrent sandboxes?

Through the XFS reflink copy-on-write mechanism implemented in CubeCoW, CubeSandbox reduces per-instance memory overhead to approximately 25 MiB for a 2 vCPU / 2 GiB configuration. This allows a single 375 GiB host to run 1,000 sandboxes while consuming only ~25 GiB of total overhead, maximizing available RAM for AI model inference.

What is the difference between snapshot creation and create-from-snapshot latency?

Snapshot creation latency scales with dirty memory size (~47 ms for clean sandboxes, plus ~40 ms per 100 MiB of dirty data), whereas create-from-snapshot operations complete in 60–85 ms regardless of snapshot size. This O(1) restoration time occurs because CubeSandbox uses CoW pointers rather than copying full data blocks during creation.

How does the eBPF-based networking (CubeVS) improve AI agent workload performance?

CubeVS replaces traditional iptables-based NAT with eBPF programs that execute at kernel speed, eliminating the per-packet processing overhead and connection tracking limits that typically constrain high-connection AI agents. This architecture, detailed in docs/zh/architecture/overview.md, provides line-speed throughput and microsecond-scale packet processing while maintaining per-sandbox network isolation.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}