# How OpenSandbox Uses Sandbox Pooling to Eliminate Cold-Start Latency

> Discover how OpenSandbox sandbox pooling slashes cold-start latency from seconds to milliseconds by pre-warming pods. Learn its benefits and implementation.

- Repository: [Alibaba/OpenSandbox](https://github.com/alibaba/OpenSandbox)
- Tags: performance
- Published: 2026-03-08

---

**OpenSandbox's sandbox pooling mechanism maintains a configurable buffer of pre-warmed pods to eliminate image-pull and container-creation delays, reducing cold-start latency from seconds to milliseconds.**

OpenSandbox implements **sandbox pooling** as a first-class Kubernetes resource to solve the cold-start problem inherent in serverless container execution. By keeping a standing army of initialized pods ready for immediate attachment to `BatchSandbox` workloads, the system bypasses the costly pod creation lifecycle that typically dominates latency in on-demand sandbox environments.

## Core Benefits of Sandbox Pooling for Cold-Start Performance

**Sandbox pooling** delivers five measurable advantages over traditional on-demand pod creation:

- **Immediate pod availability** – Idle pods in the pool are already running and can be attached to a sandbox instantly, eliminating the `kubectl create pod` → container startup delay that typically takes 5–30 seconds.
- **Image-pull avoidance** – Pre-warmed pods already have the required sandbox image cached on the node, skipping the network-bound image download that often accounts for 50–80% of cold-start time.
- **Reduced scheduling latency** – The controller binds sandboxes to existing pods rather than executing the full scheduler flow, bypassing scheduler queuing and node selection algorithms.
- **Predictable latency SLAs** – Operators configure `BufferMin` and `BufferMax` parameters to guarantee a minimum number of warm pods, ensuring consistent sub-second startup times regardless of traffic spikes.
- **Efficient resource reuse** – Multiple sandboxes share the same underlying pool infrastructure without repeatedly allocating new compute resources, lowering CPU/memory churn and improving cluster utilization.

## Sandbox Pool Architecture and CRD Design

OpenSandbox extends the Kubernetes API with a custom **Pool CRD** defined in [`kubernetes/apis/sandbox/v1alpha1/pool_types.go`](https://github.com/alibaba/OpenSandbox/blob/main/kubernetes/apis/sandbox/v1alpha1/pool_types.go). This resource declaratively specifies both the sandbox template and capacity constraints.

### Pool CRD Definition

The `PoolSpec` struct combines a pod template with capacity configuration:

```go
// https://github.com/alibaba/OpenSandbox/blob/main/kubernetes/apis/sandbox/v1alpha1/pool_types.go
type PoolSpec struct {
    Template     *corev1.PodTemplateSpec `json:"template"`
    CapacitySpec CapacitySpec            `json:"capacitySpec"`
}

```

### CapacitySpec Configuration

The `CapacitySpec` struct provides four integer fields to fine-tune pool behavior:

```go
type CapacitySpec struct {
    BufferMax int32 `json:"bufferMax"` // Maximum warm-idle pods maintained
    BufferMin int32 `json:"bufferMin"` // Guaranteed warm-idle pods (SLA floor)
    PoolMax   int32 `json:"poolMax"`   // Hard limit of total pods (warm + allocated)
    PoolMin   int32 `json:"poolMin"`   // Soft floor of total pods
}

```

These parameters allow operators to trade off resource cost against latency requirements. `BufferMin` ensures critical workloads always find a ready pod, while `PoolMax` prevents unbounded resource consumption during traffic surges.

## Implementation Details of the Pool Controller

The **PoolReconciler** in [`kubernetes/internal/controller/pool_controller.go`](https://github.com/alibaba/OpenSandbox/blob/main/kubernetes/internal/controller/pool_controller.go) implements a continuous control loop that maintains the desired pool state. It watches three object types: `Pool` (the primary resource), `Pod` (owned pool members), and `BatchSandbox` (consumers requesting pool allocation).

### Reconciliation Loop

The `Reconcile` method implements a six-step workflow:

```go
// https://github.com/alibaba/OpenSandbox/blob/main/kubernetes/internal/controller/pool_controller.go
func (r *PoolReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) {
    // 1️⃣ Load the Pool CR
    // 2️⃣ List pool's Pods and related BatchSandboxes
    // 3️⃣ Call scheduleSandbox → Allocator decides which pod serves which sandbox
    // 4️⃣ Persist allocation annotations (Allocator.PersistPoolAllocation)
    // 5️⃣ Compute a revision hash from the PodTemplate (calculateRevision)
    // 6️⃣ Update pool status & perform scaling (create / delete Pods)
}

```

### Revision Tracking and Rolling Updates

To support template updates without disrupting running workloads, the controller implements **revision tracking** via the `calculateRevision` function:

```go
func (r *PoolReconciler) calculateRevision(pool *sandboxv1alpha1.Pool) (string, error) {
    template, _ := json.Marshal(pool.Spec.Template)
    rev := sha256.Sum256(template)
    return hex.EncodeToString(rev[:8]), nil
}

```

This generates an 8-character hexadecimal hash from the JSON-serialized `PodTemplate`. Pods store this value in the label `sandbox.opensandbox.io/pool-revision`. During reconciliation, pods with mismatched revisions are considered stale and gracefully terminated, while new pods launch with the current revision—enabling zero-downtime rolling updates of pool configurations.

### Dynamic Scaling Logic

The controller calculates desired capacity using a weighted buffer formula:

```go
// https://github.com/alibaba/OpenSandbox/blob/main/kubernetes/internal/controller/pool_controller.go#L68-L80
desiredBufferCnt := (pool.Spec.CapacitySpec.BufferMin + pool.Spec.CapacitySpec.BufferMax) / 2
desiredTotalCnt := allocatedCnt + supplyCnt + desiredBufferCnt
// enforce PoolMin/PoolMax limits …

```

This logic maintains the buffer at the midpoint between `BufferMin` and `BufferMax` while respecting hard limits. When demand exceeds supply, the controller prioritizes allocating existing idle pods over creating new ones, ensuring sub-millisecond attachment times for incoming requests.

### Allocation Strategy and Annotations

The `Allocator` interface (located in `kubernetes/internal/controller/strategy/`) computes bidirectional mappings between pods and sandboxes. Allocation results persist as Kubernetes annotations on the `BatchSandbox` object using the `AnnoAllocReleaseKey` constant. The pool controller reads these annotations to distinguish between allocated (busy) and available (idle) pods, enabling accurate capacity planning without requiring pod state modifications.

## Practical Usage: Defining and Consuming Pools

Implementing **sandbox pooling** requires three artifacts: a pool definition, a sandbox request referencing the pool, and client SDK configuration.

### Creating a Pool Resource

Define a pool with specific image and capacity constraints:

```yaml
apiVersion: sandbox.opensandbox.io/v1alpha1
kind: Pool
metadata:
  name: pool-sample
spec:
  template:
    spec:
      containers:
      - name: sandbox
        image: sandbox-registry.cn-zhangjiakou.cr.aliyuncs.com/opensandbox/code-interpreter:v1.0.1
        resources:
          limits:
            memory: "512Mi"
            cpu: "500m"
  capacitySpec:
    bufferMin: 2        # Always keep at least 2 warm pods ready

    bufferMax: 5        # Do not exceed 5 idle pods when quiet

    poolMin: 2          # Minimum total pods in the pool

    poolMax: 10         # Upper bound for scaling under load

```

*Source:* [`kubernetes/test/e2e/testdata/pool-basic.yaml`](https://github.com/alibaba/OpenSandbox/blob/main/kubernetes/test/e2e/testdata/pool-basic.yaml)

### Requesting Pre-Warmed Pods via BatchSandbox

Reference the pool in your sandbox workload using `poolRef`:

```yaml
apiVersion: sandbox.opensandbox.io/v1alpha1
kind: BatchSandbox
metadata:
  name: batchsandbox-pool-sample
spec:
  poolRef: pool-sample     # Attach to the pre-warmed pool

  replicas: 3
  expireTime: "2026-12-03T12:55:41Z"

```

*Source:* [`kubernetes/config/samples/sandbox_v1alpha1_pooled_batchsandbox.yaml`](https://github.com/alibaba/OpenSandbox/blob/main/kubernetes/config/samples/sandbox_v1alpha1_pooled_batchsandbox.yaml)

### Client SDK Integration

When using the Python SDK, pass the pool reference via the `extensions` parameter:

```python

# https://github.com/alibaba/OpenSandbox/blob/main/examples/code-interpreter/main_use_pool.py

sandbox = await Sandbox.create(
    image,
    connection_config=config,
    extensions={"poolRef": "pool-sample"},   # Request pre-warmed pod allocation

    entrypoint=["/opt/opensandbox/code-interpreter.sh"],
)

async with sandbox:
    interpreter = await CodeInterpreter.create(sandbox=sandbox)
    result = await interpreter.codes.run(
        "print('hello')", language=SupportedLanguage.PYTHON
    )

```

The SDK transparently handles the allocation handshake, attaching your code execution environment to an idle pod from the specified pool rather than waiting for new pod creation.

## Summary

- **Sandbox pooling** eliminates cold-start latency by maintaining a buffer of running pods ready for immediate attachment.
- The **Pool CRD** ([`pool_types.go`](https://github.com/alibaba/OpenSandbox/blob/main/pool_types.go)) defines pre-warmed templates and capacity constraints via `BufferMin`, `BufferMax`, `PoolMin`, and `PoolMax` fields.
- The **PoolReconciler** ([`pool_controller.go`](https://github.com/alibaba/OpenSandbox/blob/main/pool_controller.go)) continuously scales pools, tracks revisions using SHA-256 hashing, and binds sandboxes via the `Allocator` strategy pattern.
- **Revision tracking** enables zero-downtime updates of pool templates through rolling pod replacements.
- Workloads consume pools by setting `spec.poolRef` in `BatchSandbox` resources or passing `{"poolRef": "name"}` in SDK extensions.

## Frequently Asked Questions

### How does sandbox pooling improve cold-start performance compared to on-demand pod creation?

**Sandbox pooling** removes the image-pull and container-initialization phases from the critical path. According to the OpenSandbox source code, pre-warmed pods in [`kubernetes/internal/controller/pool_controller.go`](https://github.com/alibaba/OpenSandbox/blob/main/kubernetes/internal/controller/pool_controller.go) are already running and have the sandbox image cached locally, reducing startup time from seconds to milliseconds.

### What happens when a pool's PodTemplate is updated?

The `PoolReconciler` detects template changes through the `calculateRevision` function in [`pool_controller.go`](https://github.com/alibaba/OpenSandbox/blob/main/pool_controller.go), which computes a SHA-256 hash of the serialized template. Pods with mismatched revision labels are gradually terminated and replaced with new pods matching the updated specification, implementing a rolling update strategy that maintains buffer capacity throughout the transition.

### How does the controller decide how many warm pods to maintain?

The scaling logic computes `desiredBufferCnt` as the average of `BufferMin` and `BufferMax` (see lines 68–80 of [`pool_controller.go`](https://github.com/alibaba/OpenSandbox/blob/main/pool_controller.go)). This value is added to the current allocation count to determine the total desired pods, then clamped between `PoolMin` and `PoolMax` constraints to prevent resource exhaustion while ensuring SLA compliance.

### Can multiple BatchSandboxes share the same pool simultaneously?

Yes. The `Allocator` interface in `kubernetes/internal/controller/strategy/` manages multi-tenancy by tracking which pods are allocated to which sandboxes using the `AnnoAllocReleaseKey` annotation. As long as the pool maintains idle pods above `BufferMin`, concurrent sandbox requests receive immediate pod assignments without contention.