How to Achieve Sub-60ms Cold Start Performance in CubeSandbox

CubeSandbox achieves sub-60ms cold start latency and approximately 5MiB memory overhead per instance by combining pre-snapshotted templates, reflink-based copy-on-write cloning, and a minimal Rust-VMM restore path that eliminates traditional VM boot sequences.

CubeSandbox is a high-performance serverless sandbox runtime developed by TencentCloud that delivers sub-60ms cold start performance for isolated AI agents and microservices. Unlike conventional virtual machines that execute full boot sequences, CubeSandbox restores execution state from pre-built memory snapshots, enabling thousands of isolated workloads to spin up on a single commodity node with minimal resource overhead.

The Three Architectural Pillars of Sub-60ms Cold Start Performance

CubeSandbox’s sub-60ms cold start capability rests on three tightly integrated techniques that work together to bypass traditional virtualization bottlenecks.

Pre-Snapshotted Templates Eliminate VM Boot

When a new sandbox is created, CubeSandbox does not boot a fresh VM from an OCI image. Instead, it restores a memory snapshot taken from a pre-built template image. The template is generated once through a pipeline that converts an OCI image to a rootfs, performs a cold boot, captures the memory state, and registers the result for reuse. This eliminates the kernel initialization, init system startup, and filesystem mounting that typically consume hundreds of milliseconds in conventional containers or VMs.

According to the architecture documentation in docs/architecture/overview.md, this template creation flow transforms a standard container image into a frozen execution state that can be resurrected instantly.

Fast Snapshot-Cloning via reflink

New sandboxes are instantiated using reflink cloning on copy-on-write capable filesystems. This operation duplicates only metadata references to the snapshot’s block device, making sandbox creation O(1) regardless of image size. The VM’s memory is instantly populated with the template’s state, while the underlying COW layer handles subsequent writes without copying the entire image.

The performance benchmark documentation in docs/guide/performance-benchmark.md details how this mechanism enables linear scaling until the node’s memory limit is reached.

Optimised Rust-VMM Restore Path

CubeSandbox uses a trimmed-down Rust VMM located in hypervisor/vmm that restores guest memory in a single, lock-free pass. The restore code deliberately skips device probing, floppy mounting, and ISO loading, running entirely in userspace to minimize context switches. The hot path for cold starts lives in hypervisor/vmm/src/memory_manager.rs, where the system creates a new memory file and immediately fills it from the snapshot without additional allocation overhead.

Performance Metrics and Resource Overhead

The sub-60ms cold start performance is measurable and consistent across deployment scenarios. Based on the public benchmark data in docs/guide/introduction.md and docs/guide/performance-benchmark.md, CubeSandbox delivers:

• Cold-start latency: < 60 ms average, with P95 ≈ 90 ms and P99 ≈ 137 ms
• Memory overhead per sandbox: ≈ 5 MiB (minimal runtime structures only)
• Maximum concurrent creations: > 500 sandboxes on a single node with linear scaling

The memory overhead remains low because the snapshot image is read-only for the majority of the sandbox lifetime. Only dirty pages are materialized in memory, limiting the resident set size. Additionally, the VMM strips optional devices like VirtIO-blk and serial consoles during cold start, reducing the state that must be restored.

Implementation Deep Dive

Understanding the code paths reveals why CubeSandbox maintains such low overhead while hitting sub-60ms targets.

Memory Manager Cold Start Path

The core restoration logic resides in hypervisor/vmm/src/memory_manager.rs. When triggering a cold start, this file handles the creation of a new memory file that is immediately populated from the template snapshot. This implementation avoids the gradual memory allocation patterns typical of booting operating systems, instead mapping the entire working set in a single operation.

Storage Layer Fallback Logic

The Cubelet storage layer in Cubelet/storage/local.go determines whether to use a cached snapshot or fall back to a cold start. This deliberate fallback mechanism ensures that the fast path remains short and predictable, only executing the full cold start routine when a cached template is unavailable. The coldStart boolean propagates through the filesystem layer, as seen in Cubelet/plugins/cbri/cubeboxcbri/virtiofs.go, ensuring that virtualization options are tuned appropriately for restoration versus boot scenarios.

Practical Implementation Examples

Below are concrete implementations showing how to trigger the sub-60ms cold start path using CubeSandbox’s official SDKs and API.

Python SDK Implementation

Create a sandbox from a template using the Python SDK to trigger the optimized cold start:

from cubesandbox import CubeClient

# Initialize client pointing to Cube API service

client = CubeClient(endpoint="http://localhost:8080")

# Select pre-built template (triggers snapshot restore)

template_name = "python-3.10-template"

# Create sandbox - executes reflink clone + memory restore

sandbox = client.create_sandbox(
    name="my-agent-01",
    template=template_name,
    resources={"cpu": 0.5, "memory": "128Mi"}
)

print(f"Sandbox ID: {sandbox.id}")
print(f"Cold-start latency (ms): {sandbox.metrics.cold_start_ms}")

This example is adapted from sdk/python/README.md, which documents the create_sandbox method and its return of cold_start_ms metrics.

HTTP API Direct Access

For direct integration without SDKs, use the REST API endpoint defined in hypervisor/vmm/src/api/openapi/cloud-hypervisor.yaml:

curl -X POST http://localhost:8080/v1/sandboxes \
  -H "Content-Type: application/json" \
  -d '{
        "name": "my-agent-01",
        "template": "python-3.10-template",
        "resources": { "cpu": 0.5, "memory": "128Mi" }
      }' | jq .

The response includes a cold_start_ms field reporting the actual latency of the snapshot-restore operation, explicitly noted in the OpenAPI specification’s "cold" parameter descriptions.

Go SDK with Latency Measurement

Measure cold start latency programmatically using the Go SDK:

package main

import (
    "context"
    "fmt"
    "time"
    cubesandbox "github.com/TencentCloud/CubeSandbox/sdk/go"
)

func main() {
    client := cubesandbox.NewClient("http://localhost:8080")
    start := time.Now()
    
    sandbox, err := client.CreateSandbox(context.Background(),
        cubesandbox.SandboxSpec{
            Name:     "go-agent-01",
            Template: "go-1.20-template",
            Resources: map[string]string{
                "cpu":    "0.5",
                "memory": "128Mi",
            },
        })
    if err != nil {
        panic(err)
    }
    
    elapsed := time.Since(start).Milliseconds()
    fmt.Printf("Sandbox %s created in %d ms (reported cold start: %d ms)\n",
        sandbox.ID, elapsed, sandbox.Metrics.ColdStartMs)
}

This implementation from sdk/go/README.md demonstrates how to capture both client-side and server-reported cold start metrics.

Summary

• Pre-snapshotted templates eliminate traditional VM boot sequences by restoring memory state from frozen templates rather than booting from OCI images.
• Reflink-based COW cloning creates new sandboxes in O(1) time by duplicating metadata rather than copying data blocks, enabling instant memory population.
• Minimal Rust-VMM in hypervisor/vmm/src/memory_manager.rs executes lock-free memory restoration without device probing, keeping the restore path under 60ms.
• Storage layer intelligence in Cubelet/storage/local.go ensures cold start logic only executes when cached snapshots are unavailable.
• Measurable performance delivers <60ms average cold start with ~5MiB memory overhead per instance, supporting >500 concurrent sandboxes per node.

Frequently Asked Questions

How does CubeSandbox achieve sub-60ms cold starts compared to Firecracker or gVisor?

CubeSandbox achieves sub-60ms cold starts by restoring pre-snapshotted memory templates rather than booting a kernel and init system from scratch. While Firecracker optimizes the boot sequence and gVisor provides user-space kernel emulation, CubeSandbox bypasses boot entirely by reflinking a memory snapshot and restoring it through a minimal Rust-VMM. This architectural difference eliminates the initialization overhead inherent in traditional microVMs.

What filesystem requirements are necessary for the reflink cloning mechanism?

The reflink-based snapshot cloning requires a copy-on-write capable filesystem such as Btrfs or XFS with reflink support enabled. This allows CubeSandbox to create new sandbox instances by duplicating only metadata references to the template snapshot, achieving O(1) creation time regardless of the image size. Without reflink support, the system would fall back to full copy operations, significantly increasing cold start latency.

Can custom OCI images be used with CubeSandbox's sub-60ms cold start feature?

Yes, custom OCI images can be converted into pre-snapshotted templates through the template creation pipeline described in docs/architecture/overview.md. The process involves converting the OCI image to a rootfs, performing an initial cold boot to reach the ready state, capturing the memory snapshot, and registering the result. Once templatized, any sandbox created from that image inherits the sub-60ms cold start performance.

What is the actual memory overhead per sandbox instance?

CubeSandbox maintains approximately 5 MiB of memory overhead per sandbox instance. This minimal footprint comes from the fact that the snapshot image remains read-only for the majority of the sandbox lifetime, with only dirty pages materialized in memory. The VMM also strips optional devices during the cold start restore path, further reducing the runtime memory footprint compared to traditional virtual machines.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}