Debugging Sandbox Provisioning and Command Execution in OpenSandbox: A Complete Guide

OpenSandbox provides three distinct debugging layers—server-side Python logs, Go execd hooks via ExecuteResultHook, and SDK HTTP debug mode—to trace issues from Kubernetes pod creation through final command output.

OpenSandbox is a multi-layered system where a Python FastAPI server coordinates sandbox lifecycle, a Go execution daemon (execd) runs code inside isolated containers, and language-specific SDKs expose convenient APIs to callers. Debugging sandbox provisioning or command execution in OpenSandbox requires understanding how to activate observability at each of these three layers.

Server-Side Debugging for Sandbox Provisioning

The OpenSandbox server uses Python’s standard logging module to surface provisioning states, Kubernetes polling events, and Docker image handling details.

Kubernetes Service Polling and Log Configuration

The core logic for waiting on sandbox readiness resides in server/src/services/k8s/kubernetes_service.py within the _wait_for_sandbox_ready method. When the log level is set to DEBUG, this method emits detailed polling messages that help identify where provisioning stalls.

To enable debug logging, modify your server configuration file (typically ~/.sandbox.toml or the example.config.toml in the server/ directory):

# ~/.sandbox.toml

[log]
level = "DEBUG"

With debug logging active, you will see output such as:

DEBUG:root:Workload not found yet for sandbox 4f2c9a1b…
INFO:root:Sandbox 4f2c9a1b state: Pending - Pulling image
INFO:root:Sandbox 4f2c9a1b state: Running - IP assigned 10.1.2.3

These messages originate from the polling loop in kubernetes_service.py between lines 73-77, where the code checks for workload existence and emits a debug line when the pod has not yet materialized. If a sandbox never reaches the "Running" state, these logs pinpoint whether the issue lies in pod creation, image pulling, or scheduling.

Docker Image Handling Debug Logs

For deployments using the Docker backend, server/src/services/docker.py contains additional debug instrumentation. For example, when the daemon falls back to a cached image, it logs this event at the debug level around lines 809-810, helping you trace image-related failures.

Execd-Level Debugging with ExecuteResultHook

The execd component handles the actual code execution lifecycle inside containers. Debugging at this layer involves the ExecuteResultHook interface defined in components/execd/pkg/runtime/types.go.

Understanding the Hook Interface

The ExecuteResultHook struct provides callback functions that the daemon invokes during execution:

type ExecuteResultHook struct {
    OnExecuteInit     func(context string)
    OnExecuteResult   func(result map[string]any, count int)
    OnExecuteStatus   func(status string)
    OnExecuteStdout   func(stdout string) //nolint:predeclared
    OnExecuteStderr   func(stderr string) //nolint:predeclared
    OnExecuteError    func(err *execute.ErrorOutput)
    OnExecuteComplete func(executionTime time.Duration)
}

If no hook is supplied, execd uses a default implementation that prints to STDOUT. Developers can replace this with custom logic for richer logging or test assertions.

Implementing Custom Debug Hooks

To debug command execution issues, inject custom hooks when building the execution request in components/execd/pkg/web/controller/codeinterpreting.go (around lines 240-251 where ExecuteCodeRequest is constructed):

req := &runtime.ExecuteCodeRequest{
    Code: "print('hello from execd')",
    Hooks: runtime.ExecuteResultHook{
        OnExecuteStdout: func(out string) {
            log.Infof("[Execd stdout] %s", out)
        },
        OnExecuteStderr: func(err string) {
            log.Warnf("[Execd stderr] %s", err)
        },
        OnExecuteError: func(e *execute.ErrorOutput) {
            log.Errorf("[Execd error] %s – %s", e.EName, e.EValue)
        },
    },
}
if err := codeRunner.Execute(req); err != nil {
    log.Fatalf("Execution failed: %v", err)
}

Because the daemon only prints to STDOUT when hooks are nil, adding these callbacks enriches observability without changing execution semantics.

SDK-Level Debugging for HTTP Request Tracing

Language-specific SDKs provide a debug flag that enables HTTP-level request and response logging, making it easy to trace why a sandbox creation call failed.

Enabling Debug Mode in Python SDK

In the Python SDK, the debug flag is defined in sdks/sandbox/python/src/opensandbox/config/connection.py (lines 64-66). When enabled, the internal HTTP client (using httpx) logs request URLs, headers, body, and response status.

from opensandbox import Sandbox, Config

cfg = Config(
    endpoint="http://localhost:8000",
    debug=True,  # Enable HTTP-level debugging

)

sandbox = Sandbox(cfg)
sandbox.create(image="python:3.10")
sandbox.exec("print('debugging SDK')")

Any 4xx/5xx errors from the FastAPI server will be echoed to stdout with full request context, allowing you to distinguish between network issues, authentication failures, and server-side provisioning errors.

End-to-End Debugging Checklist

Combine these three layers to trace any issue from sandbox provisioning through final command output:

1. Configure server logs: Set log.level = "DEBUG" in ~/.sandbox.toml to see Kubernetes polling messages from kubernetes_service.py.
2. Enable SDK tracing: Set debug=True in your SDK configuration to capture HTTP request/response cycles.
3. Inject execd hooks: Implement custom ExecuteResultHook callbacks to capture stdout, stderr, and execution errors from the Go daemon.
4. Correlate with cluster events: Use kubectl get pod -n <namespace> -l sandbox_id=<id> -w to correlate server debug lines with actual Kubernetes events.

Summary

• Server-side debugging relies on Python's logging module configured via ~/.sandbox.toml, with key insights in server/src/services/k8s/kubernetes_service.py for provisioning states and server/src/services/docker.py for image handling.
• Execd debugging uses the ExecuteResultHook interface defined in components/execd/pkg/runtime/types.go, allowing custom callbacks for stdout, stderr, and errors during command execution.
• SDK debugging is enabled via the debug=True flag in connection configurations (e.g., sdks/sandbox/python/src/opensandbox/config/connection.py), providing HTTP-level request and response tracing.

Frequently Asked Questions

How do I enable debug logging in the OpenSandbox server?

Set the log level to DEBUG in your server configuration file (typically ~/.sandbox.toml or example.config.toml):

[log]
level = "DEBUG"

This activates detailed output from server/src/services/k8s/kubernetes_service.py, including polling messages like "Workload not found yet for sandbox …" that help diagnose provisioning delays.

What is the ExecuteResultHook in OpenSandbox's execd component?

ExecuteResultHook is a struct defined in components/execd/pkg/runtime/types.go that provides callback functions for execution lifecycle events, including OnExecuteStdout, OnExecuteStderr, and OnExecuteError. By implementing custom hooks when constructing an ExecuteCodeRequest in components/execd/pkg/web/controller/codeinterpreting.go, developers can capture real-time output and errors for logging or testing purposes.

How can I debug HTTP requests between my application and OpenSandbox?

Enable the debug flag in your SDK's connection configuration. For the Python SDK, set debug=True in opensandbox.config.connection.Config (defined in sdks/sandbox/python/src/opensandbox/config/connection.py). This activates HTTP client logging that dumps request URLs, headers, bodies, and server responses to stdout, making it easy to identify authentication failures, timeouts, or malformed requests.

Where can I find logs related to Docker image pulling in OpenSandbox?

Docker-related debug logs are located in server/src/services/docker.py. When the server falls back to a cached image or encounters pull issues, it emits debug-level messages around lines 809-810. Ensure your server log level is set to DEBUG in ~/.sandbox.toml to capture these events, which help diagnose image availability and caching problems during sandbox provisioning.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}