How the iii-sandbox Worker Provides Ephemeral VM Execution Environments from OCI Rootfs

The iii-sandbox worker launches as an external child process, caches OCI rootfs images, and boots them inside libkrun micro-VMs with ephemeral overlays to provide isolated, short-lived execution environments.

The iii-sandbox worker is an external worker implementation in the iii-hq/iii repository that transforms standard OCI container images into lightweight, ephemeral virtual machines. Unlike traditional container runtimes, this worker extracts only the root filesystem, caches it locally, and executes commands inside hardware-virtualized micro-VMs through an overlay-based storage system.

Architecture Overview

When a driver's config.yaml specifies the worker class iii-sandbox, the III engine resolves this via the known-external table in engine/src/workers/external.rs. The engine spawns the binary iii-worker sandbox-daemon as a child process, piping stdout and stderr to prevent terminal contamination. This daemon registers 14 distinct triggers—including sandbox::create, sandbox::exec, sandbox::stop, and filesystem operations—through the III SDK entry point in crates/iii-worker/src/sandbox_daemon/mod.rs.

The architecture separates concerns across three layers:

• Orchestration layer: The engine manages worker lifecycle through ExternalWorker::start_background_tasks
• Daemon layer: The sandbox daemon handles OCI resolution, caching, and VM orchestration
• Virtualization layer: libkrun provides KVM-based micro-VMs with custom init systems

The Lifecycle of an Ephemeral Sandbox

Worker Launch and Registration

The lifecycle begins when ExternalWorker::start_background_tasks spawns iii-worker sandbox-daemon. The daemon immediately registers its API surface through functions like register_sandbox_create, register_sandbox_exec, and register_sandbox_stop in crates/iii-worker/src/sandbox_daemon/mod.rs. This registration makes the sandbox triggers available to the III SDK for remote invocation.

Image Resolution and Caching

When a client invokes sandbox::create, the daemon executes handle_create in crates/iii-worker/src/sandbox_daemon/create.rs. This process validates the request against an image allow-list, then resolves the image name to an OCI reference via the catalog module.

The rootfs handling follows a strict caching strategy:

1. Check for a cached unpacked rootfs at ~/.iii/cache/<slug>/
2. If missing and auto_install is enabled, pull the OCI image using crates/iii-worker/src/sandbox_daemon/auto_install.rs
3. Extract and cache the filesystem for future reuse

This separation means the daemon never runs a full container engine—it only requires the extracted filesystem blob.

VM Boot Process

After caching, handle_create constructs an ephemeral overlay directory structure (merged, upper, and work directories) via overlay::OverlayLayout in crates/iii-worker/src/sandbox_daemon/overlay.rs. The daemon then invokes IiiWorkerLauncher::boot from crates/iii-worker/src/sandbox_daemon/adapters.rs to launch the micro-VM.

The boot sequence executes iii-worker __vm-boot (the same binary, re-executed) with flags specifying:

• Path to the cached rootfs (read-only)
• Overlay paths (writable upper layer)
• CPU and memory limits
• Unix sockets for control (control.sock) and command execution (shell.sock)

First-boot provisioning includes platform-specific setup: macOS code-signing and extraction of libkrunfw (the KVM hypervisor runtime) plus the init.krun binary via ensure_libkrunfw and ensure_init_binary. The launcher polls shell.sock until a successful connection confirms the VM is alive before returning the sandbox ID to the caller.

Command Execution

Once booted, clients invoke sandbox::exec handled by handle_exec in crates/iii-worker/src/sandbox_daemon/exec.rs. The daemon opens a ShellRunner that communicates with the running VM over shell.sock using the iii-shell-client protocol.

Commands execute inside the overlay's merged view, inheriting environment variables set at boot time plus any per-execution env overrides. Output streams back to the caller with a 1 MiB cap per stream (stdout and stderr separately).

Filesystem Operations

The daemon provides ten filesystem triggers (sandbox::fs::ls, write, read, rm, etc.) implemented in crates/iii-worker/src/sandbox_daemon/fs.rs. These manipulate the overlay's upper and work layers using standard POSIX calls through iii-filesystem. Because each sandbox maintains an isolated overlay, all mutations disappear when the sandbox stops, ensuring true ephemerality.

Cleanup and Reaping

Resource cleanup operates through two mechanisms:

Idle reaping: A background task (run_reaper_loop in crates/iii-worker/src/sandbox_daemon/reaper.rs) monitors last_exec_at timestamps. Sandboxes exceeding idle_timeout_secs (default 300 seconds) receive automatic termination and overlay deletion.

Explicit stopping: The sandbox::stop trigger (handled by handle_stop in crates/iii-worker/src/sandbox_daemon/stop.rs) sends SIGTERM, waits 200ms, then escalates to SIGKILL. The overlay directory is then deleted, freeing RAM and CPU while leaving the cached rootfs intact for reuse.

Key Implementation Details

OCI Rootfs Handling

According to the source code in crates/iii-worker/src/sandbox_daemon/auto_install.rs, the auto-install logic pulls images only when missing from cache and validates them against the allow-list. This design ensures fast sandbox creation for subsequent uses of the same image while maintaining security boundaries.

Overlay Filesystem

The overlay::OverlayLayout creates a unique directory triad for each sandbox:

• upper/: Captures all write operations
• work/: Required for overlay filesystem internals
• merged/: The unified view presented to the VM

Reads fallback to the immutable OCI rootfs when files are absent from the upper layer. When sandbox::stop executes, the daemon recursively deletes these three directories, instantly reverting the filesystem state.

Resource Constraints

The daemon enforces hierarchical limits defined in the configuration:

• Per-image caps: default_cpus, default_memory_mb, and per_image_caps restrict individual sandboxes
• Global limits: max_concurrent_sandboxes prevents resource exhaustion
• Error handling: Requests exceeding limits return S-code S400 before VM allocation occurs

Code Examples

TypeScript SDK

import { registerWorker } from 'iii-sdk';

const iii = registerWorker('ws://127.0.0.1:49134');

// Boot a sandbox from the built-in "python" image
const { sandbox_id } = await iii.trigger({
  function_id: 'sandbox::create',
  payload: { image: 'python', cpus: 1, memory_mb: 512 },
  timeoutMs: 300_000,
});

// Run a command inside the VM
const out = await iii.trigger({
  function_id: 'sandbox::exec',
  payload: {
    sandbox_id,
    cmd: 'python3',
    args: ['-c', 'print(2 + 2)'],
  },
  timeoutMs: 35_000,
});
console.log(out.stdout); // → "4\n"

// Clean up
await iii.trigger({
  function_id: 'sandbox::stop',
  payload: { sandbox_id, wait: true },
});

Python SDK

from iii import register_worker

iii = register_worker('ws://127.0.0.1:49134')

# Create ephemeral VM

res = await iii.trigger({
    "function_id": "sandbox::create",
    "payload": {"image": "python", "cpus": 1, "memory_mb": 512},
    "timeout_ms": 300_000,
})
sandbox_id = res["sandbox_id"]

# Execute command

out = await iii.trigger({
    "function_id": "sandbox::exec",
    "payload": {"sandbox_id": sandbox_id,
                "cmd": "python3",
                "args": ["-c", "print(2 + 2)"]},
    "timeout_ms": 35_000,
})
print(out["stdout"])   # → "4\n"

# Terminate sandbox

await iii.trigger({
    "function_id": "sandbox::stop",
    "payload": {"sandbox_id": sandbox_id, "wait": True},
})

CLI Shortcut

iii sandbox run python -- python3 -c 'print(2 + 2)'

# prints "4" and automatically stops the VM

Summary

• External process model: The iii-sandbox worker runs as iii-worker sandbox-daemon, spawned by the engine via ExternalWorker::start_background_tasks in engine/src/workers/external.rs.
• OCI to VM pipeline: The daemon resolves OCI references, caches unpacked rootfs at ~/.iii/cache/<slug>/, and boots them using IiiWorkerLauncher::boot with libkrun virtualization.
• Ephemeral storage: Each sandbox receives an isolated overlay (upper/work/merged directories) that is deleted on stop, while the cached rootfs remains for reuse.
• Socket-based communication: Commands flow through shell.sock using the iii-shell-client protocol, with output capped at 1 MiB per stream.
• Automatic lifecycle management: Idle sandboxes exceed idle_timeout_secs (default 300s) are reaped by run_reaper_loop, and explicit stops trigger SIGTERM → SIGKILL sequences.

Frequently Asked Questions

How does iii-sandbox differ from Docker or containerd?

Unlike Docker or containerd, the iii-sandbox worker does not run a container engine or manage daemon processes. According to the implementation in crates/iii-worker/src/sandbox_daemon/create.rs, it extracts only the OCI rootfs layer, caches it locally, and boots a libkrun micro-VM with that filesystem mounted read-only. Commands execute inside hardware-virtualized environments rather than kernel namespaces, providing stronger isolation boundaries while maintaining sub-second startup times for cached images.

Where are the OCI images cached and how is storage managed?

The daemon stores unpacked rootfs directories under ~/.iii/cache/<slug>/ as implemented in crates/iii-worker/src/sandbox_daemon/auto_install.rs. These cached layers persist across sandbox restarts for fast subsequent creation. However, each sandbox instance receives its own ephemeral overlay directory (created by overlay::OverlayLayout in crates/iii-worker/src/sandbox_daemon/overlay.rs) that is deleted when the sandbox stops via handle_stop in crates/iii-worker/src/sandbox_daemon/stop.rs. This design separates immutable image data from mutable sandbox state.

What happens if a sandbox exceeds resource limits?

The daemon validates requests against default_cpus, default_memory_mb, per_image_caps, and max_concurrent_sandboxes before booting. Requests exceeding these limits immediately return S-code S400 without spawning a VM, as defined in the resource enforcement logic. For sandboxes that exceed idle time, the run_reaper_loop in crates/iii-worker/src/sandbox_daemon/reaper.rs automatically terminates them after idle_timeout_secs (default 300 seconds) of inactivity.

Can I run custom OCI images or only built-in ones?

The sandbox supports custom OCI images when configured with an allow-list. The handle_create function in crates/iii-worker/src/sandbox_daemon/create.rs validates image names against this list before resolution. If auto_install is enabled in the configuration, the daemon automatically pulls, extracts, and caches unauthorized but valid OCI references using the catalog module and auto_install.rs logic. Images must provide a standard Linux rootfs compatible with libkrun's virtualization layer.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}