CubeAPI Gateway Architecture for High-Concurrency REST API Handling

CubeSandbox implements a four-layer Rust-based gateway—comprising the Cube API (Axum), Cube Proxy (tokio), Nginx front-end, and optional sub-domain isolation—to handle millions of concurrent REST requests with per-key token-bucket rate limiting and minimal latency.

The TencentCloud/CubeSandbox project delivers a high-performance API gateway engineered specifically for sandboxed AI agent workloads and E2B-compatible environments. This CubeAPI gateway architecture for high-concurrency REST API handling leverages Rust's async runtime and a layered reverse-proxy design to ensure strict tenant isolation, sub-millisecond latency, and horizontal scalability out of the box.

The Four-Layer Architecture

The gateway consists of four tightly integrated layers that process requests from the public internet down to the individual sandbox instances.

Layer 1: Cube API (E2B-Compatible REST Service)

The Cube API serves as the core HTTP service implementing all sandbox, template, snapshot, and cluster operations. In CubeAPI/src/routes.rs, the Axum framework mounts all routes under the /cubeapi/v1 prefix, creating a unified routing table for sandbox management endpoints.

The implementation uses Rust with the Axum web framework to maximize throughput. In CubeAPI/src/middleware/rate_limit.rs, a per-API-key token-bucket rate limiter inspects the X-API-Key header (falling back to IP-based anonymous tracking) and immediately returns 429 Too Many Requests when quotas are exceeded, preventing abuse before requests reach business logic.

Layer 2: Cube Proxy (High-Performance Reverse-Proxy)

The Cube Proxy operates as a minimal, high-performance reverse-proxy that exposes public endpoints and forwards them to the internal Cube API process. Defined in CubeAPI/src/main.rs, this component runs as a tiny Rust HTTP server using tokio for single-threaded, async connection multiplexing.

By default, the proxy listens on 0.0.0.0:3000 (configurable via CUBE_API_URL) and forwards all /:path traffic directly to the Cube API socket. This design avoids extra network hops, using the host's loopback interface for direct inter-process communication.

Layer 3: Nginx Front-End (One-Click Deployment)

For production deployments, an Nginx container provides TLS termination, static asset serving for the Web UI, and reverse-proxy capabilities. The configuration in deploy/one-click/webui/nginx.conf exposes port 12088 for the human-facing interface while proxying /cubeapi/ requests to the host-gateway (host.docker.internal).

The Nginx layer handles TLS certificate management and serves the compiled Web UI (webui/dist), offloading encryption overhead from the Rust API stack while maintaining a clean separation between static content and dynamic API traffic.

Layer 4: Gateway Sub-Domain Isolation (Optional)

For advanced isolation scenarios, the architecture supports per-sandbox sub-domains using the format <port>-<sandboxId>.<domain>. When configured, the Web UI stores a gatewayDomain for each instance (see web/src/api/client.ts lines 46-52), constructing URLs like https://12088-<sandboxId>.cube.app/....

This pattern, configured through web/src/components/agents/AgentSettingsDialog.tsx (lines 56-66), requires a wildcard DNS record and TLS certificate but ensures complete traffic isolation per sandbox, preventing cross-tenant WebSocket interference and enabling per-sandbox rate limit enforcement.

Request Flow and Rate Limiting

Understanding the request lifecycle clarifies how the CubeAPI gateway architecture maintains performance under high concurrency:

1. Incoming request arrives at Nginx (port 12088) and is reverse-proxied via /cubeapi/ to the Cube Proxy (port 3000).
2. Cube Proxy forwards the request to the Cube API process over the host's loopback interface, avoiding NAT overhead.
3. Cube API extracts the X-API-Key header and checks the token-bucket rate limiter in CubeAPI/src/middleware/rate_limit.rs. Exceeded quotas trigger an immediate 429 response.
4. Passed requests dispatch through the Axum router in CubeAPI/src/routes.rs to the appropriate handler (e.g., sandbox creation, template listing).
5. For assistant-level gateways, the UI appends the sub-domain (<port>-<sandboxId>.<gatewayDomain>), allowing the backend to enforce per-sandbox limits based on the origin's API key.

Design Rationale for High Concurrency

The architecture addresses specific high-concurrency challenges through targeted technical decisions:

• High request volume: Async Rust (tokio) combined with a single-threaded proxy eliminates thread-per-connection overhead, using non-blocking I/O to sustain millions of concurrent connections.
• Latency reduction: Direct host-gateway routing via Docker's host.docker.internal bridge eliminates network address translation hops, keeping internal latency sub-millisecond.
• Tenant isolation: Per-sandbox sub-domains with independent rate-limiting buckets ensure that a single tenant cannot saturate the global request queue, protecting multi-tenant workloads.
• Operational security: Nginx handles TLS termination and static file serving, offloading cryptographic operations from the high-performance Rust API layer while maintaining battle-tested SSL/TLS implementations.

Practical Usage Examples

Access the gateway directly through the Cube Proxy for local development:

# Basic health check (no API key required)

curl http://localhost:3000/cubeapi/v1/health

# Authenticated request with rate-limit key

curl -H "X-API-Key: my-key-123" \
     http://localhost:3000/cubeapi/v1/sandboxes

When utilizing gateway domain isolation, route traffic through the sub-domain endpoint:

curl https://3000-<sandbox-id>.cube.app/cubeapi/v1/health

Both routes terminate at the same CubeAPI backend, but the sub-domain variant provides strict CORS isolation and WebSocket separation per sandbox instance.

Summary

• CubeSandbox's gateway uses a four-layer architecture (Cube API, Cube Proxy, Nginx, sub-domain isolation) to handle high-concurrency REST traffic.
• Rust and Axum power the core API with async tokio networking, while token-bucket rate limiting in CubeAPI/src/middleware/rate_limit.rs enforces per-key quotas.
• Direct host-gateway routing minimizes latency by avoiding NAT hops between the proxy and API layers.
• Optional sub-domain isolation (<port>-<sandboxId>.<domain>) enables per-sandbox origin separation and independent rate limiting.
• Nginx front-end handles TLS termination and static assets, offloading encryption from the high-throughput Rust stack.

Frequently Asked Questions

How does CubeAPI prevent a single tenant from overwhelming the gateway?

The gateway implements a token-bucket rate limiter in CubeAPI/src/middleware/rate_limit.rs that tracks requests per X-API-Key header. When a key exceeds its configured quota, the middleware immediately returns HTTP 429 before the request reaches the Axum router, ensuring that abusive traffic never impacts other tenants or consumes compute resources.

What makes the Cube Proxy more efficient than a traditional reverse-proxy?

The Cube Proxy runs as a minimal Rust service using tokio for single-threaded, async connection multiplexing. Unlike traditional thread-per-connection proxies, it maintains state for thousands of concurrent connections without context-switching overhead, forwarding traffic directly to the Cube API via the host's loopback interface to eliminate network hops.

Why does the architecture include both a Cube Proxy and Nginx?

Nginx handles TLS termination and static file serving for the Web UI, leveraging battle-tested SSL implementations and disk I/O optimization. The Cube Proxy focuses solely on high-performance API request forwarding, avoiding the overhead of TLS handshakes and file system operations. This separation allows the Rust-based API layer to dedicate CPU cycles to business logic while Nginx manages edge connectivity.

How does sub-domain isolation improve security for sandboxed environments?

Sub-domain isolation (configured via gatewayDomain in web/src/api/client.ts) assigns each sandbox a unique origin (<port>-<sandboxId>.<domain>). This prevents cross-origin resource sharing (CORS) conflicts and ensures that WebSocket sessions for different sandboxes never share a domain, blocking cross-site scripting (XSS) vectors and enabling per-sandbox rate limit enforcement based on the unique sub-domain's API key.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}