How AutoPause and AutoResume Suspend Idle Sandboxes in TencentCloud CubeSandbox

The AutoPause and AutoResume mechanism uses a shared Nginx dictionary to track sandbox states and a Lua rewrite-phase gate to automatically suspend idle containers and reactivate them on the next request without client intervention.

The AutoPause and AutoResume feature in TencentCloud CubeSandbox automates resource conservation by suspending idle sandboxes and transparently reactivating them when new traffic arrives. This mechanism is implemented through CubeProxy, an Nginx-based sidecar that manages the lifecycle of each sandbox instance using shared dictionaries and Lua scripting. The implementation spans configuration files and Lua modules that coordinate state transitions across the running, pausing, and paused states.

Shared State Storage in Nginx

The foundation of the AutoPause system rests on a worker-shared dictionary declared in CubeProxy/nginx.conf. This memory zone provides fast, consistent state access across all Nginx workers without requiring external RPC calls.

# CubeProxy/nginx.conf

lua_shared_dict cube_sandbox_state 10m;   # sandbox_id → "running" | "pausing" | "paused"

The scheduler writes state transitions into this dictionary via the admin API, while the rewrite-phase gate reads values on every incoming request. A companion dictionary, cube_last_active, tracks per-sandbox activity timestamps for idle detection.

Rewrite-Phase Gate Logic

The CubeProxy/lua/sandbox_state.lua module implements the _M.gate(ins_id) function, which executes during the Nginx rewrite phase. This gate intercepts every request and determines whether to allow traffic, send a retry signal, or trigger an automatic resume.

Running State

When the shared dictionary returns "running" or nil (default), the gate permits the request to flow normally to the upstream sandbox container.

Pausing State

If the state is "pausing", the gate returns a 503 Service Unavailable response with a Retry-After: 30 header. This drains in-flight traffic while the sidecar persists container state, preventing new connections from reaching the sandbox during the shutdown sequence.

-- CubeProxy/lua/sandbox_state.lua
if state == "pausing" then
    ngx.header["Retry-After"] = 30
    ngx.exit(ngx.HTTP_SERVICE_UNAVAILABLE)
end

Paused State and Auto-Resume

When the state is "paused", the gate automatically contacts the sidecar to resume the container before allowing the request to proceed:

-- CubeProxy/lua/sandbox_state.lua
if state == "paused" then
    local ok, err = ngx.location.capture("/__resume", { method = ngx.HTTP_POST })
    if not ok then
        ngx.log(ngx.ERR, "failed to resume sandbox ", ins_id, ": ", err)
        ngx.exit(ngx.HTTP_INTERNAL_SERVER_ERROR)
    end
    return
end

The /__resume location, defined in nginx.conf as an internal endpoint, proxies to the CubeSidecar service that handles the actual container startup. Once the container resumes, the updated state reflects "running" and the original request continues transparently.

Admin API for State Management

The CubeProxy/lua/admin_phase.lua module exposes HTTP endpoints that allow the CubeMaster scheduler to orchestrate pauses. The handle_state() function validates and applies state transitions:

-- CubeProxy/lua/admin_phase.lua
local function handle_state()
    local body = read_json_body()
    local sandbox_id = body.sandbox_id
    local st = body.state   -- "running" | "pausing" | "paused"

    if not (st == "running" or st == "pausing" or st == "paused") then
        return reply_error(400, "state must be one of running|pausing|paused")
    end

    local ok, err = ngx.shared.cube_sandbox_state:set(sandbox_id, st)
    if not ok then
        return reply_error(500, "failed to set state: " .. (err or "unknown"))
    end

    reply(200, '{"msg":"ok"}')
end

The scheduler typically implements a two-phase pause: first writing "pausing" to allow the gate to drain requests, then writing "paused" after a grace period once the container is fully suspended.

Activity Tracking for Idle Detection

The CubeProxy/lua/log_phase.lua module records request metadata to support idle detection decisions. The record_activity(ins_id) function updates a timestamp in the shared dictionary:

-- CubeProxy/lua/log_phase.lua
local function record_activity(ins_id)
    local key = "last_active:" .. ins_id
    ngx.shared.cube_last_active:set(key, ngx.now())
end

The CubeMaster scheduler polls these timestamps to identify sandboxes that have exceeded the configured idle timeout threshold, triggering the admin API to initiate the pause sequence.

Implementation Examples

Triggering a Pause from the Scheduler

# Phase 1: Mark as pausing to drain traffic

curl -X POST http://cube-proxy-admin/api/state \
     -d '{"sandbox_id":"sb-1234","state":"pausing"}'

# Phase 2: After grace period, mark as paused

curl -X POST http://cube-proxy-admin/api/state \
     -d '{"sandbox_id":"sb-1234","state":"paused"}'

Automatic Resume on First Request

When a client sends a request to a paused sandbox:

GET http://sandbox-1234.example.com/api/status

The gate detects the paused state, calls the sidecar resume endpoint, and transparently forwards the request once the container is active. The client receives a normal 200 response without handling any interruption.

Inspecting Current State

curl http://cube-proxy-admin/api/state?sandbox_id=sb-1234

# Response: {"sandbox_id":"sb-1234","state":"paused"}

Summary

• CubeProxy implements AutoPause and AutoResume through a combination of Nginx shared dictionaries and Lua scripting.
• The cube_sandbox_state dictionary stores lifecycle states (running, pausing, paused) that are checked on every request in CubeProxy/lua/sandbox_state.lua.
• The rewrite-phase gate returns 503 Retry-After during the pausing state and automatically triggers resume via ngx.location.capture when in the paused state.
• The CubeProxy/lua/admin_phase.lua module provides the HTTP interface for the scheduler to initiate state transitions.
• Activity tracking in CubeProxy/lua/log_phase.lua feeds idle detection algorithms that determine when to suspend sandboxes.

Frequently Asked Questions

How does the gate handle requests while a sandbox is pausing?

When the state is "pausing", the _M.gate(ins_id) function in CubeProxy/lua/sandbox_state.lua returns an HTTP 503 response with a Retry-After: 30 header. This signals clients to retry later while preventing new traffic from reaching the container during the shutdown grace period.

What happens to the first request that hits a paused sandbox?

The gate automatically detects the "paused" state and executes ngx.location.capture("/__resume") to signal the CubeSidecar to start the container. The request waits while the sidecar restores the sandbox, then proceeds normally. This resume process is transparent to the client, which receives the expected response once the container is active.

Where does the scheduler get the idle metrics to decide when to pause?

The scheduler reads from the cube_last_active shared dictionary, which is populated by CubeProxy/lua/log_phase.lua. Each request triggers an update to the last_active:{sandbox_id} key with the current timestamp, allowing the scheduler to calculate elapsed idle time and trigger pauses for inactive sandboxes.

Can the pause state be set directly without going through the "pausing" intermediate state?

While the handle_state() function in CubeProxy/lua/admin_phase.lua technically accepts direct transitions to "paused", the scheduler typically uses the two-phase approach (pausing then paused) to ensure the gate has time to drain in-flight requests and return 503 errors to new connections before the container fully suspends.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}