# How AutoResume Works for Idle Sandbox Optimization in CubeSandbox

> Discover how AutoResume optimizes idle sandboxes in CubeSandbox. Learn how it intercepts requests, locks operations, and restores VMs to prevent duplicate work and boost performance.

- Repository: [Tencent Cloud/CubeSandbox](https://github.com/TencentCloud/CubeSandbox)
- Tags: internals
- Published: 2026-07-09

---

**AutoResume wakes paused sandboxes on-demand by intercepting incoming requests in CubeProxy, acquiring a distributed Redis lock to serialize resume operations, and restoring the VM via CubeMaster RPC while coalescing concurrent requests to prevent duplicate work.**

The CubeSandbox platform from TencentCloud implements an intelligent resource management system that automatically pauses idle sandboxes and rapidly resumes them when new traffic arrives. Understanding how AutoResume works for idle sandbox optimization reveals a sophisticated distributed coordination mechanism that balances millisecond-scale wake-up times with strict consistency guarantees across multiple side-car replicas.

## The AutoResume Trigger Flow

When a sandbox exceeds its `timeout_seconds` threshold, the **Sweeper** component marks it as paused in Redis by setting `cube:v1:shared:sandbox:lifecycle:state:<sandbox-id>` to `"paused"`. At this point, the side-car proxy (`CubeProxy`) stops forwarding traffic. The resume sequence initiates when a new dataplane request targets a paused sandbox.

### CubeProxy Interception and Internal Routing

The request first hits the **Lua gate** ([`sandbox_state.lua`](https://github.com/TencentCloud/CubeSandbox/blob/main/sandbox_state.lua)) within CubeProxy. Instead of forwarding to the paused sandbox, the gate returns HTTP 503 and simultaneously dispatches an internal request to `POST /internal/resume` to trigger the wake-up sequence. This logic is implemented in the router at [`network-agent/internal/service/cube_router.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/network-agent/internal/service/cube_router.go).

### Side-car HTTP Handler

The side-car's HTTP server receives the resume request at [`cube-lifecycle-manager/internal/httpapi/server.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/cube-lifecycle-manager/internal/httpapi/server.go). The handler extracts the sandbox ID and delegates to the Resumer package:

```go
func (s *Server) resumeHandler(w http.ResponseWriter, r *http.Request) {
    sandboxID := chi.URLParam(r, "sandbox_id")
    err := s.resumer.Resume(r.Context(), sandboxID)
    if err != nil {
        http.Error(w, err.Error(), http.StatusServiceUnavailable)
        return
    }
    w.WriteHeader(http.StatusOK)
}

```

## Distributed State Coordination

The Resumer must coordinate across multiple side-car replicas to ensure only one attempts the expensive VM restoration. This uses a Redis-backed state machine with four possible values: `"paused"`, `"running"`, `"pausing"`, and `"resuming"`.

### The Redis Lock Mechanism

In [`cube-lifecycle-manager/internal/resumer/resumer.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/cube-lifecycle-manager/internal/resumer/resumer.go), the `acquireResumeOwnership` function implements a GET-then-SETNX pattern:

```go
func (r *Resumer) acquireResumeOwnership(ctx context.Context, sandboxID string) error {
    cur, ok, err := r.o.Redis.GetState(ctx, sandboxID)
    if err != nil { return err }

    switch {
    case !ok || cur == "paused":
        return r.o.Redis.SetState(ctx, sandboxID, "resuming", r.o.StateLockTTL)
    case cur == "running":
        return errAlreadyRunning
    case cur == "pausing" || cur == "resuming":
        if err := r.waitForRunning(ctx, sandboxID); err != nil { return err }
        return errAlreadyRunning
    default:
        if err := r.waitForRunning(ctx, sandboxID); err != nil { return err }
        return errAlreadyRunning
    }
}

```

**Key behaviors:**

- If the state is `"paused"`, the caller sets it to `"resuming"` and owns the resume operation.
- If `"running"`, the operation short-circuits as already complete.
- If `"pausing"` or `"resuming"`, the caller waits for the peer to finish using `waitForRunning`.

## Concurrent Request Handling

Each side-car replica handles request coalescing internally to prevent duplicate RPCs when multiple simultaneous requests arrive for the same sandbox.

### Single-Flight Implementation

The `Resumer` maintains an in-process `map[string]*call` (lines 40-44 in [`resumer.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/resumer.go)) that merges concurrent Go goroutines:

```go
func (r *Resumer) Resume(ctx context.Context, sandboxID string) error {
    r.mu.Lock()
    if c, ok := r.calls[sandboxID]; ok {
        r.mu.Unlock()
        select {
        case <-c.done: return c.err
        case <-ctx.Done(): return ctx.Err()
        }
    }
    c := &call{done: make(chan struct{})}
    r.calls[sandboxID] = c
    r.mu.Unlock()
    defer func() {
        r.mu.Lock(); delete(r.calls, sandboxID); r.mu.Unlock()
        close(c.done)
    }()
    c.err = r.doResume(ctx, sandboxID)
    return c.err
}

```

This ensures that even under high concurrency, only one resume RPC executes per replica, while others block until completion.

## VM Restoration and State Propagation

Once the Resumer acquires ownership, it invokes `CubeMaster.Resume` via the RPC client in [`cube-lifecycle-manager/internal/cubemasterclient/client.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/cube-lifecycle-manager/internal/cubemasterclient/client.go). This triggers the VM-restore path through CubeMaster → Cubelet → containerd, reconstructing the sandbox from its pause snapshot.

### Success Bookkeeping

After successful restoration (or detecting an already-running state), the Resumer performs three critical updates:

1. **Redis State**: Writes `"running"` back to the lifecycle state key with `SetState`
2. **Proxy Synchronization**: Pushes the new state to all CubeProxy replicas via `ProxyPush.SetState`
3. **Activity Tracking**: Updates `registry.LastActiveMs` to prevent immediate re-pausing by the Sweeper

```go
func (r *Resumer) doResume(ctx context.Context, sandboxID string) error {
    // ... after successful RPC or already-running case ...
    _ = r.o.Redis.SetState(ctx, sandboxID, "running", r.o.StateLockTTL)
    _ = r.o.ProxyPush.SetState(ctx, sandboxID, "running")
    r.o.Registry.MergeLastActive(sandboxID, time.Now().UnixMilli())
    r.o.Log.Info("auto-resumed sandbox", zap.String("sandbox_id", sandboxID))
    return nil
}

```

## AutoResume Configuration and Safety

The AutoResume feature is controlled per-sandbox via the `AutoResume` flag in `SandboxLifecycleMeta`. If disabled, the Resumer returns `auto_resume not enabled for sandbox` (lines 99-101 in [`resumer.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/resumer.go)) and the request fails fast.

The system handles edge cases gracefully:

- **Lock expiration**: If the Redis lock expires without reaching a terminal state, `waitForRunning` returns an error, the caller returns 503, and the next request retries the sequence.
- **Idempotent RPCs**: The `callCubeMasterResume` function treats "already-in-state" responses as success, making retries safe.

## Summary

- **AutoResume triggers** when CubeProxy intercepts traffic to a paused sandbox and invokes the side-car's `/internal/resume` endpoint.
- **Distributed coordination** uses a Redis state machine (`paused`, `pausing`, `resuming`, `running`) with atomic GET-SETNX operations to serialize resume attempts across replicas.
- **Request coalescing** via an in-process `calls` map ensures single-flight execution within each side-car instance, preventing duplicate CubeMaster RPCs.
- **State propagation** updates Redis, pushes changes to all CubeProxy instances, and refreshes the activity timestamp to prevent immediate re-pausing.
- **Safety mechanisms** include per-sandbox feature flags, idempotent RPC handling, and graceful 503 fallbacks when locks expire.

## Frequently Asked Questions

### How does AutoResume prevent multiple side-car replicas from duplicating resume work?

The system uses **Redis as a distributed lock** with a state machine. When a replica calls `acquireResumeOwnership`, it attempts to atomically transition the state from `"paused"` to `"resuming"`. Only the replica that succeeds owns the resume operation. Others wait via `waitForRunning` or short-circuit if the state is already `"running"`. This is implemented in [`cube-lifecycle-manager/internal/resumer/resumer.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/cube-lifecycle-manager/internal/resumer/resumer.go).

### What happens if a resume request fails or the Redis lock expires?

If the lock expires before completion, `waitForRunning` returns an error, and the Resumer returns HTTP 503 to the caller. The original dataplane request (blocked in Nginx with `proxy_read_timeout`) may retry, or the next incoming request will restart the resume sequence. This makes the system self-healing without manual intervention.

### Where is the AutoResume feature flag configured?

The `AutoResume` boolean resides in `SandboxLifecycleMeta` (defined in [`cube-lifecycle-manager/internal/lifecycle/schema.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/cube-lifecycle-manager/internal/lifecycle/schema.go)). When the Resumer receives a resume request, it checks `entry.Meta.AutoResume` at lines 99-101 of [`resumer.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/resumer.go). If false, it immediately returns an error, preventing automatic wake-up for sandboxes that require manual management.

### How does the system handle concurrent requests to the same paused sandbox?

Within each side-car replica, the `Resumer` maintains a `calls` map that implements a **single-flight pattern**. The first goroutine for a sandbox ID executes the actual resume logic, while subsequent concurrent callers block on a channel until the operation completes. This prevents duplicate RPCs to CubeMaster while allowing distributed coordination via Redis across different replicas.