How AutoPause and AutoResume Work for Idle Sandbox Suspension and Wake-on-Request in CubeSandbox

CubeSandbox employs two complementary sidecar components—a sweeper for automatic idle detection and a resumer for on-demand wake-up—to suspend inactive sandboxes and resume them instantly upon request, eliminating resource waste while maintaining availability.

TencentCloud/CubeSandbox implements an efficient lifecycle management system that automatically pauses idle sandboxes and resumes them when traffic arrives. This mechanism leverages a periodic sweeper to monitor activity timestamps and a request-driven resumer to restore state, working together to minimize resource consumption without manual intervention.

Architecture Overview

The AutoPause and AutoResume features rely on metadata stored in the sandbox registry upon creation. When a sandbox starts, CubeMaster registers it with lifecycle metadata including auto_pause, auto_resume, and timeout_seconds in the registry (cube-lifecycle-manager/internal/registry/registry.go). This metadata schema is defined in cube-lifecycle-manager/internal/lifecycle/schema.go and mirrored in CubeMaster/pkg/service/sandbox/types/types.go.

Two distinct components handle state transitions:

• Sweeper: Runs on a ticker interval (default 5 seconds) to detect idle sandboxes
• Resumer: Handles incoming requests to paused sandboxes via the /internal/resume endpoint

AutoPause Implementation

The idle suspension logic resides in cube-lifecycle-manager/internal/sweeper/sweeper.go. The sweeper evaluates each sandbox's eligibility for pausing based on configurable timeouts and activity timestamps.

Idle Detection Logic

The sweeper wakes every Options.Interval (default 5 seconds) and iterates through the registry. For each entry, it calculates a baseline timestamp representing the most recent activity:

baseline = max(LastActiveMs, CreatedAt)

The idle duration is computed as now - baseline. The sweeper skips entries within the bootstrap warm-up window to avoid pausing sandboxes that have not yet reported initial activity.

The Pause Decision

If Meta.AutoPause is true and the idle duration exceeds the sandbox's TimeoutSeconds (or default idle timeout), the sweeper invokes tryPause. This method:

1. Acquires a state lock in Redis using SETNX on cube:v1:shared:sandbox:lifecycle:state:<id>
2. Sends the CubeMaster.Pause RPC
3. On success, writes "paused" to Redis and notifies CubeProxy via ProxyPush

If AutoPause is disabled (false), the sweeper kills the sandbox instead of pausing it.

// From sweeper.go: deciding whether to pause or kill
if e.Meta.AutoPause {
    // idleFor > timeout → pause
    s.tryPause(ctx, e)
}

AutoResume Implementation

When a paused sandbox receives an incoming request, CubeProxy intercepts the traffic and POSTs to the resumer's /internal/resume endpoint. The resumer logic lives in cube-lifecycle-manager/internal/resumer/resumer.go.

Request Interception and Validation

The Resume method first validates the sandbox's configuration:

if !entry.Meta.AutoResume {
    return errors.New("auto_resume not enabled for sandbox")
}

If AutoResume is disabled, the request aborts immediately with an error.

Lock Acquisition and RPC

The resumer coalesces concurrent resume attempts through Redis locking. The acquireResumeOwnership function obtains the "resuming" lock using the same Redis key pattern but with a different lock value. Once ownership is granted, callCubeMasterResume sends the RPC to CubeMaster.

After a successful resume, the resumer updates three critical states:

1. Redis state: Set to "running" via SetState
2. CubeProxy state: Updated via ProxyPush.SetState
3. Registry timestamp: LastActiveMs updated to current time

// Successful resume sequence
if err := r.callCubeMasterResume(ctx, sandboxID, entry.Meta.InstanceType); err != nil {
    return err
}
// Updates state and timestamp

Configuration and Code Examples

Configure sandbox lifecycle management at creation time using the schema defined in cube-lifecycle-manager/internal/lifecycle/schema.go:

// Example: configuring a sandbox to auto-pause after 5 min and allow auto-resume
sandboxReq := lifecycle.CreateSandboxRequest{
    SandboxID:       "sbx-123",
    AutoPause:       true,
    AutoResume:      true,
    TimeoutSeconds:  lifecycle.TimeoutSecondsPtr(300), // 5 min
}

The Python example in examples/code-sandbox-quickstart/auto-resume.py demonstrates client-side interaction with auto-resuming sandboxes.

Error Handling and Edge Cases

Pause Failures

If the CubeMaster.Pause RPC fails, the sweeper increments pauseFailed metrics and logs a warning. The next ticker interval will retry the pause operation automatically.

Resume Conflict Resolution

The resumer handles several CubeMaster response codes:

• NotFound: The sandbox is evicted from all caches (Redis and CubeProxy)
• AlreadyInState: Treated as success; the sandbox is already running
• Real errors: Clear the resume lock so subsequent requests can retry

This ensures that transient failures do not permanently block sandbox availability.

Summary

• AutoPause uses a periodic sweeper (sweeper.go) to monitor idle time against LastActiveMs and CreatedAt, pausing sandboxes via RPC when timeouts exceed configured thresholds
• AutoResume intercepts requests through CubeProxy, validates the AutoResume flag, acquires Redis locks, and restores sandbox state via callCubeMasterResume
• Both features rely on registry metadata (SandboxLifecycleMeta) and Redis state management (cube:v1:shared:sandbox:lifecycle:state:<id>)
• Disabled AutoPause triggers sandbox termination rather than suspension
• The system handles concurrent resume attempts through ownership locks and treats AlreadyInState responses as successful wake-ups

Frequently Asked Questions

What happens if AutoPause is disabled on a sandbox?

When Meta.AutoPause is false, the sweeper bypasses the pause logic and instead calls the kill routine. The sandbox is terminated rather than suspended, freeing resources but requiring a cold start on the next request.

How does CubeSandbox handle concurrent resume requests?

The resumer implements a locking mechanism via acquireResumeOwnership that uses Redis to grant exclusive "resuming" rights to one request. Other concurrent requests wait until the lock is released or the operation completes, preventing duplicate RPCs to CubeMaster.

What is the default idle timeout for AutoPause?

While the specific default value depends on the cluster configuration, the sweeper uses the TimeoutSeconds field from the sandbox metadata. If unset, it falls back to a system default idle timeout configured in the lifecycle manager options.

How does the sweeper avoid pausing sandboxes that are still booting?

The sweeper implements a bootstrap warm-up window check that skips entries until they have reported initial activity or exceeded the warm-up threshold. This prevents premature pausing of sandboxes that have not yet set their LastActiveMs timestamp.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}