How to Customize CubeSandbox Behavior Through Configuration

You customize CubeSandbox by editing YAML files under configs/single-node/ and setting environment variables for the NGINX-based CubeProxy, allowing you to control API endpoints, resource quotas, scheduling policies, and auto-pause/resume behavior.

CubeSandbox is a multi-component sandbox platform from TencentCloud that manages containerized workloads across CubeMaster, Cubelet, and CubeProxy components. To customize CubeSandbox behavior through configuration, you modify structured YAML files for the control plane and data plane workers, alongside environment variables for the reverse proxy. These configuration sources determine everything from REST API timeouts to Redis coordination and per-node resource limits.

CubeMaster Configuration (cubemaster.yaml)

The CubeMaster control plane reads its runtime parameters from configs/single-node/cubemaster.yaml. This file is organized into distinct sections that govern HTTP service settings, logging, default sandbox templates, and storage backends.

HTTP API and Logging Settings

The common section controls where the REST API listens and how it handles idle connections.

• http_port: Defaults to 8089; change this to bind the API to a different port.
• http_bind: Set to 0.0.0.0 to listen on all interfaces or a specific IP for restricted access.
• Time-out values: http_readtimeout, http_writetimeout, and http_idletimeout guard against slow clients.
• sync_meta_data_interval and sync_metric_data_interval dictate how frequently CubeMaster pulls state from Redis.

The log section configures the logging backend. Adjust path, file_size, file_num, and level (e.g., info or debug) to integrate with your observability pipeline.

Default Sandbox Parameters

The cubelet_conf section defines default parameters injected into every Cubelet when CubeMaster spawns a sandbox.

• default_timeout_insec: Set to -1 for no global idle timeout, or specify seconds to enforce maximum sandbox idle time.
• create_timeout_insec, create_concurrent_limit, and destroy_concurent_limit: Tune RPC deadlines and concurrency caps to prevent overwhelming nodes during burst creates.
• enable_exposed_port and exposed_port_list: Open additional host ports to the sandbox (e.g., 80 or 443).

Storage and Scheduler Policies

The ossdb_config and instance_db_config sections contain MySQL connection details for the metadata store. Replace placeholder values like __CUBE_SANDBOX_MYSQL_*__ with your actual credentials.

The redis section configures the central coordination store:

• nodes: Host and port of Redis.
• node_metric_ttl_sec: How long a node's health metric lives without heartbeat (default 600 seconds).
• sandbox_proxy_ttl_sec: TTL for routing keys; set to 0 to disable refresh.

The scheduler section defines placement logic:

• priority_select_num and metric_update_timeout control the scheduling loop cadence.
• filter.enable_filters: Activate node-level filters such as cpu and mem.
• overcommit_ratio: Global CPU and memory over-commit factors (default cpu = 3, mem = 2).

Cubelet Configuration (cubelet.yaml)

Cubelet runs on each worker host and executes sandbox containers, reading its settings from configs/single-node/cubelet.yaml.

Resource Quotas and GC Settings

The host.quota section restricts per-node resource consumption:

• mcpu_limit, mem_limit, and mvm_limit: Set to 0 or leave empty for unlimited resources.
• paused_resource_release_ratio: Fraction of resources released when a sandbox is paused (default 0.0, meaning keep full quota).

The host.gc section controls garbage collection of cached images and code snapshots:

• code_expiration_time: Default "72h" for code snapshots.
• image_expiration_time: Default "24h" for cached images.

The common section specifies the network_agent_endpoint (typically /tmp/cube/network-agent-grpc.sock) for communication with the Network Agent.

Network Agent Configuration (network-agent.yaml)

The Network Agent is a lightweight gRPC server that forwards network-related RPCs between Cubelet and CubeMaster. Its configuration in configs/single-node/network-agent.yaml includes:

• listen: UNIX domain socket for the Cubelet side.
• health_listen: HTTP endpoint for health checks.
• grpc_listen: gRPC socket used by CubeMaster's cubelet_conf.network_agent_endpoint.

Changing these socket paths enables integration with custom init-systems or container-based deployments.

CubeProxy Environment Variables and NGINX Settings

CubeProxy is an NGINX-based reverse proxy that routes external traffic to individual sandboxes. Unlike other components, it relies primarily on environment variables declared in env directives within CubeProxy/nginx.conf.

Proxy Registration and Heartbeat

The Lua scripts read these variables at worker start:

• CUBE_PROXY_REGISTRY_ENABLE: Controls whether the proxy registers itself in Redis.
• CUBE_PROXY_RESUME_URL: The side-car address for auto-pause/resume (e.g., http://127.0.0.1:9093/internal/resume).
• CUBE_PROXY_HEARTBEAT_INTERVAL_MS: Controls heartbeat frequency.
• CUBE_PROXY_REGISTRY_REDIS_HOST, CUBE_PROXY_REGISTRY_REDIS_PORT, CUBE_PROXY_REGISTRY_REDIS_PASSWORD, CUBE_PROXY_REGISTRY_REDIS_DB: Redis connection parameters for registration.

Lua Shared Memory and Routing

The lua_shared_dict sections in nginx.conf allocate in-memory caches for sandbox metadata, state, and last-active timestamps. Adjust these sizes if you anticipate more than 100,000 concurrent sandboxes.

The Lua rewrite phase (lua/path_rewrite_phase.lua) injects sandbox IDs and target ports into requests before proxying to the upstream backend.

Practical Configuration Examples

Change the CubeMaster REST API Port

To move the API from port 8089 to 9090:

# configs/single-node/cubemaster.yaml

common:
  http_port: 9090
  http_bind: 0.0.0.0

Restart the cubemaster service to apply changes.

Enable Built-in Authentication

# configs/single-node/cubemaster.yaml

auth:
  enable: true

After restart, requests must include Authorization: Bearer <token> headers.

Limit Concurrent Sandbox Creation

# configs/single-node/cubemaster.yaml

cubelet_conf:
  create_concurrent_limit: 50
  destroy_concurent_limit: 50

This prevents RPC overload during burst traffic.

Reduce Node Detection Time

To detect dead nodes faster, lower the Redis TTL:

# configs/single-node/cubemaster.yaml

redis:
  node_metric_ttl_sec: 300

Configure CubeProxy Side-car Address

Set the environment variable before starting NGINX:

export CUBE_PROXY_RESUME_URL="http://127.0.0.1:9093/internal/resume"

Or declare it in CubeProxy/nginx.conf:

env CUBE_PROXY_RESUME_URL;

Enable Exposed Host Ports

Allow sandboxes to bind to ports 80 and 443:

# configs/single-node/cubemaster.yaml

cubelet_conf:
  enable_exposed_port: true
  exposed_port_list:
    - "80"
    - "443"

Summary

• CubeMaster behavior is controlled via configs/single-node/cubemaster.yaml, covering HTTP APIs, logging, MySQL/Redis storage, and scheduling policies.
• Cubelet settings in configs/single-node/cubelet.yaml manage resource quotas, GC intervals, and pause-resume ratios.
• Network Agent socket paths in configs/single-node/network-agent.yaml enable custom intra-node communication setups.
• CubeProxy relies on environment variables (defined in CubeProxy/nginx.conf) for Redis registration, heartbeat intervals, and auto-pause/resume URLs.
• Always restart the affected component after modifying configuration files to ensure changes take effect.

Frequently Asked Questions

Where are CubeSandbox configuration files located?

All component-specific YAML files reside under configs/single-node/ in the repository root. CubeProxy uses CubeProxy/nginx.conf for NGINX directives and environment variable declarations. These files are loaded at startup by their respective binaries.

How do I change the default sandbox idle timeout?

Edit the cubelet_conf section in configs/single-node/cubemaster.yaml and set default_timeout_insec to the desired number of seconds. A value of -1 disables the timeout entirely, while positive values enforce automatic termination after the specified idle period.

Can I customize resource over-commit ratios?

Yes. In configs/single-node/cubemaster.yaml, locate the scheduler section and modify overcommit_ratio. The default values are cpu = 3 and mem = 2, meaning the scheduler assumes three times the available CPU and twice the available memory can be allocated safely.

How does CubeProxy auto-pause functionality work?

CubeProxy uses Lua scripts (lua/path_rewrite_phase.lua) that read the CUBE_PROXY_RESUME_URL environment variable to determine where to forward resume requests for paused sandboxes. When a request arrives for a paused sandbox, the proxy sends a resume signal to this address before proxying the traffic, enabling cold-start optimization.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}