# Troubleshooting Steps for Common CubeSandbox Creation and Startup Failures

> Resolve CubeSandbox creation and startup failures with expert troubleshooting steps. Debug template validation, network conflicts, and host mount permissions efficiently.

- Repository: [Tencent Cloud/CubeSandbox](https://github.com/TencentCloud/CubeSandbox)
- Tags: how-to-guide
- Published: 2026-07-11

---

**Diagnose CubeSandbox creation failures by checking template validation in [`CubeMaster/pkg/service/sandbox/sandbox_create.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/CubeMaster/pkg/service/sandbox/sandbox_create.go), network CIDR conflicts in [`hostdir_mount.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/hostdir_mount.go), and host-mount permissions in [`Cubelet/storage/files.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/Cubelet/storage/files.go), then verify Cubelet connectivity via the CBR-I health endpoint.**

When working with the **CubeSandbox** repository from TencentCloud, sandbox creation and startup failures typically occur during the orchestration between **Cubemaster** and **Cubelet** components. Understanding the specific **troubleshooting steps for common sandbox creation and startup failures** requires tracing the request lifecycle through template validation, network allocation, and root-fs preparation. This guide provides actionable diagnostics based on the actual source code implementation to resolve these failures quickly.

## Common Failure Categories and Root Causes

Sandboxes fail when any step in the creation chain returns an error. The system categorizes these failures into six primary areas based on the code paths in [`sandbox_run.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/sandbox_run.go) and related modules.

### Template Validation Errors

The **Cubemaster** receives the `CreateSandbox` request and immediately validates the `templateID` in [`CubeMaster/pkg/service/sandbox/sandbox_create.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/CubeMaster/pkg/service/sandbox/sandbox_create.go). If the template lookup fails, the request aborts with `ErrorCode_TemplateNotFound` or `cubesandbox.ErrTemplateNotFound`. Verify template existence using the CLI before submission.

### Network CIDR Conflicts

During host selection, [`CubeMaster/pkg/service/sandbox/hostdir_mount.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/CubeMaster/pkg/service/sandbox/hostdir_mount.go) allocates a pod-wide CIDR. Duplicate subnet errors surface when this range overlaps with existing network interfaces, causing the creation to timeout. Check `ip netns list` on the host to identify conflicts.

### Host-Mount Permission Denied

File operations fail with "Permission denied" when [`Cubelet/storage/files.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/Cubelet/storage/files.go) detects insufficient privileges. The sandbox user (typically UID `sandbox`) requires read/write access to the host directory. This check occurs during the `CreateSandboxRootfsFromTemplate` phase in [`Cubelet/storage/local.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/Cubelet/storage/local.go).

### Resource Quota Violations

Immediate stops marked as "OOMKilled" or "CPU throttling" originate from [`Cubelet/storage/pool.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/Cubelet/storage/pool.go) and [`Cubelet/storage/pool_withreflink.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/Cubelet/storage/pool_withreflink.go). These modules enforce size limits during root-fs creation, rejecting over-allocated requests before the sandbox process launches.

### Cubelet Connectivity and RPC Timeouts

[`CubeMaster/pkg/service/sandbox/sandbox_timeout.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/CubeMaster/pkg/service/sandbox/sandbox_timeout.go) and [`sandbox_exec.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/sandbox_exec.go) handle RPC retries to the Cubelet via the **CBR-I (CubeBox Runtime Interface)**. Network partitions or misconfigured `CUBE_SANDBOX_DOMAIN` variables cause "cubesandbox: timeout" errors after the initial creation appears successful.

### Authentication and Proxy Failures

The Go SDK in [`sdk/go/client.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/sdk/go/client.go) injects `Authorization` headers for every request. Missing or expired tokens result in 401/403 responses from the API gateway before the request ever reaches the Cubemaster.

## Step-by-Step Debugging Workflow

Follow this systematic approach to isolate the failure point:

1. **Inspect the API response** – Error messages wrap `cubesandbox.Err*` types. Check for `cubesandbox.ErrSandboxNotFound` or `cubesandbox.ErrTemplateNotFound` in the client output.

2. **Analyze Cubemaster logs** – Search for `log.G(ctx)` traces containing `CreateSandbox`, `handleCubelet`, or `failover`. Look for the `RetCode` field to identify specific error codes like `ErrorCode_TemplateNotFound`.

3. **Validate template availability** – Run `cubemastercli template list` to confirm the template exists in the template store ([`CubeMaster/pkg/service/templatecenter/store.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/CubeMaster/pkg/service/templatecenter/store.go)).

4. **Check network allocations** – Execute `ip netns list` on the host to verify no CIDR overlaps exist with the subnet allocated by [`network-agent/internal/service/netdevice.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/network-agent/internal/service/netdevice.go).

5. **Verify host-mount permissions** – Ensure the sandbox user has write access to the host directory. Adjust permissions using `chmod` or `chown` as documented in the host-mount permissions guide.

6. **Test Cubelet connectivity** – Run `curl http://$CUBE_SANDBOX_DOMAIN/healthz` from the master node. If this fails, inspect DNS, firewall, or TLS configuration.

## Code Examples for Error Handling

Implement defensive checks in your application code to catch failures at the SDK level:

```go
// Example: Create a sandbox and handle common errors
ctx := context.Background()
client, _ := cubesandbox.NewClient(&cubesandbox.Config{
    SandboxDomain: "sandbox.internal", // must match the deployment
})

// Request a sandbox from a known template
sb, err := client.CreateSandbox(ctx, &cubesandbox.CreateSandboxRequest{
    TemplateID: "tpl-basic",
    CpuCount:   2,
    MemoryMB:   1024,
})
if err != nil {
    // Inspect the wrapped error
    if errors.Is(err, cubesandbox.ErrTemplateNotFound) {
        log.Fatalf("Template missing – run `cubemastercli template list`")
    }
    if errors.Is(err, cubesandbox.ErrSandboxNotFound) {
        log.Fatalf("Sandbox creation failed – check host‑mount permissions")
    }
    log.Fatalf("Unexpected error: %v", err)
}
log.Printf("Sandbox %s created, ID=%s", sb.TemplateID, sb.SandboxID)

// After creation, ensure the sandbox is running
info, _ := client.GetInfo(ctx, sb.SandboxID)
if info.State != "running" {
    log.Printf("Sandbox not started, state=%s – see Cubemaster logs for details", info.State)
}

```

Use these CLI commands for rapid host-side diagnostics:

```bash

# 1. Verify template existence

cubemastercli template list | grep tpl-basic

# 2. Inspect host CIDR allocations

ip netns exec $(cubemastercli host list | grep active | awk '{print $1}') ip -o -4 addr show

# 3. Test Cubelet health endpoint

curl -s http://sandbox.internal/healthz && echo "Cubelet reachable"

```

## Key Source Files for Troubleshooting

Understanding these specific files accelerates root cause analysis:

- **[`CubeMaster/pkg/service/sandbox/sandbox_run.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/CubeMaster/pkg/service/sandbox/sandbox_run.go)** – Orchestrates sandbox creation, retry logic, and metric collection via `createSandboxContext`.

- **[`CubeMaster/pkg/service/sandbox/sandbox_create.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/CubeMaster/pkg/service/sandbox/sandbox_create.go)** – Contains the `CreateSandbox` function that validates `templateID` and initiates the context.

- **[`CubeMaster/pkg/service/sandbox/hostdir_mount.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/CubeMaster/pkg/service/sandbox/hostdir_mount.go)** – Allocates pod-wide CIDRs; the source of network conflict errors.

- **[`Cubelet/storage/local.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/Cubelet/storage/local.go)** – Implements `CreateSandboxRootfsFromTemplate` to prepare the filesystem before process launch.

- **[`Cubelet/storage/files.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/Cubelet/storage/files.go)** – Performs file reads/writes inside sandboxes; surfaces permission errors to the user.

- **[`Cubelet/storage/pool.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/Cubelet/storage/pool.go)** and **[`pool_withreflink.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/pool_withreflink.go)** – Enforce resource quotas and size limits during storage allocation.

- **[`CubeMaster/pkg/service/sandbox/sandbox_timeout.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/CubeMaster/pkg/service/sandbox/sandbox_timeout.go)** – Manages RPC timeouts and retry logic for Cubelet communication.

- **[`sdk/go/client.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/sdk/go/client.go)** – Provides the Go SDK interface and authentication header injection.

- **[`network-agent/internal/service/netdevice.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/network-agent/internal/service/netdevice.go)** – Manages pod CIDR allocation and conflict detection.

## Summary

- **Template validation** occurs in [`sandbox_create.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/sandbox_create.go) and returns `ErrorCode_TemplateNotFound` when the `templateID` is invalid.
- **Network CIDR conflicts** arise from overlapping subnets in [`hostdir_mount.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/hostdir_mount.go) and require checking `ip netns` allocations.
- **Host-mount permissions** are enforced in [`Cubelet/storage/files.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/Cubelet/storage/files.go), requiring the sandbox user to have directory write access.
- **Resource quotas** are managed by [`pool.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/pool.go) and [`pool_withreflink.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/pool_withreflink.go), rejecting requests that exceed memory or CPU limits.
- **Cubelet connectivity** issues manifest as RPC timeouts in [`sandbox_timeout.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/sandbox_timeout.go) and require verifying `CUBE_SANDBOX_DOMAIN` and health endpoints.
- **Authentication failures** occur in [`sdk/go/client.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/sdk/go/client.go) when `Authorization` headers are missing or expired.

## Frequently Asked Questions

### Why does my sandbox return "template not found" during creation?

This error originates in [`CubeMaster/pkg/service/sandbox/sandbox_create.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/CubeMaster/pkg/service/sandbox/sandbox_create.go) when the `CreateSandbox` function fails to locate the requested `templateID` in the template store. Verify the template exists by running `cubemastercli template list` and ensure the ID matches exactly, including case sensitivity.

### How do I resolve network CIDR conflicts when creating sandboxes?

CIDR conflicts occur when [`CubeMaster/pkg/service/sandbox/hostdir_mount.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/CubeMaster/pkg/service/sandbox/hostdir_mount.go) attempts to allocate a subnet that overlaps with existing host interfaces. Run `ip netns list` on the host to identify occupied ranges, then reconfigure the network pool or remove conflicting interfaces before retrying the creation request.

### What causes "permission denied" errors in sandbox file operations?

The [`Cubelet/storage/files.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/Cubelet/storage/files.go) module returns this error when the sandbox user (typically UID `sandbox`) lacks read/write permissions on the mounted host directory. Check the directory ownership with `ls -la` and adjust permissions using `chown` or `chmod` to grant the sandbox user access, particularly for directories used during `CreateSandboxRootfsFromTemplate`.

### Why is my sandbox stuck in a timeout or "not found" state after creation?

This indicates RPC communication failure between Cubemaster and Cubelet, handled in [`CubeMaster/pkg/service/sandbox/sandbox_timeout.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/CubeMaster/pkg/service/sandbox/sandbox_timeout.go) and [`sandbox_exec.go`](https://github.com/TencentCloud/CubeSandbox/blob/main/sandbox_exec.go). Verify the `CUBE_SANDBOX_DOMAIN` environment variable is correctly set, test connectivity with `curl http://$CUBE_SANDBOX_DOMAIN/healthz`, and ensure no firewall rules or DNS misconfigurations block the CBR-I interface.