How to Convert OCI Images into Reusable CubeSandbox Templates

CubeSandbox converts OCI images into templates by pulling image layers via the native exporter, streaming them into a root filesystem directory, and packaging the result into an ext4 image artifact without requiring Docker or external binaries.

Converting standard container images into CubeSandbox templates allows you to instantiate lightweight, isolated sandboxes from any OCI-compliant registry. The TencentCloud/CubeSandbox repository implements this through a three-layer architecture that moves from CLI parsing to asynchronous root filesystem extraction. Understanding this conversion path helps you troubleshoot build failures and optimize template creation for production workloads.

The Three-Layer Conversion Workflow

The conversion process flows from user input through the master service to the native image exporter. Each layer handles distinct responsibilities: flag parsing, job orchestration, and low-level layer streaming.

CLI Entry Point and Request Assembly

The journey begins in CubeMaster/cmd/cubemastercli/commands/cubebox/template.go with the TemplateCreateFromImageCommand function (lines 751-770). This handler parses flags such as --image, --writable-layer-size, --network-type, and --allow-internet-access, then assembles a types.CreateTemplateFromImageReq struct.

The CLI validates registry credentials, merges network configuration overrides, and submits the request via HTTP POST to /cube/template/from-image. Upon submission, the master returns a TemplateImageJobInfo containing a job_id that the CLI uses to track progress through the runImageJobWatch polling mechanism.

Master Service Job Submission

On the master side, SubmitTemplateFromImage in CubeMaster/pkg/templatecenter/template_image.go (lines 95-105) normalizes the incoming request. This function checks for duplicate jobs, persists a new TemplateImageJob record to the database, and spawns runTemplateImageJob in a separate goroutine to handle the asynchronous work.

The master handler decouples the HTTP request from the long-running export process, allowing the client to disconnect while the server continues pulling and unpacking image layers. The job runner manages state transitions through phases: PULLING, UNPACKING, and READY.

Root Filesystem Export via Native Streamer

The core conversion logic resides in exportImageRootfs, which delegates to StreamRegistryToDir when the request specifies ExportMode: native (the default). Located in CubeMaster/pkg/templatecenter/image/native.go (lines 74-78), this function performs the actual OCI-to-rootfs translation:

• Image resolution: Calls nativeImageForSource to resolve the OCI reference into a digest and manifest.
• Concurrent downloads: Pulls layers in parallel using containerd/go-containerregistry with remote.WithAuth and layer.Compressed, respecting the nativeExportConcurrency limit.
• Streaming decompression: Pipes each compressed layer through compression.DecompressStream and applies it directly to the destination directory using archive.Apply.
• Progress reporting: Wraps download streams in a progressReader that reports bytes transferred and layer completion to the job status endpoint.
• Cleanup: Deletes temporary layer files immediately after application to minimize disk footprint.

After StreamRegistryToDir completes, the job creates an ext4 image using mkfs.ext4, stores it in the RootfsArtifact table, and updates the job status to READY for sandbox creation.

Why the Native Exporter is Preferred

CubeSandbox provides multiple export modes, but the native path offers distinct advantages for production deployments.

Zero external dependencies: Unlike the dockerExportImageRootfs or dockerlessExportImageRootfs legacy paths, the native exporter avoids shelling out to Docker, Skopeo, or Umoci binaries. This reduces the attack surface and eliminates version compatibility issues.

Concurrent prefetching: The native implementation downloads multiple layers simultaneously using the nativeExportConcurrency setting, maximizing network throughput for large images.

Atomic filesystem operations: Temporary layer files are written to the destination's parent directory and renamed into place, ensuring that os.Rename never encounters cross-device (EXDEV) errors.

Real-time progress visibility: The progressReader integration provides granular feedback (bytes, layers, speed) to the CLI watch command, enabling accurate progress bars for UI consumers.

Practical Implementation Examples

Command-Line Usage

Convert a public Ubuntu image into a CubeSandbox template with a 20GB writable layer:

cubemastercli template create-from-image \
    --image docker.io/library/ubuntu:22.04 \
    --writable-layer-size 20Gi \
    --instance-type cubebox \
    --network-type tap \
    --node 10.0.1.5 \
    --allow-internet-access \
    --expose-port 80 \
    --registry-username myuser \
    --registry-password mypass

The CLI automatically polls the job status and renders progress steps:

[1/7] PULLING progress=12%
[2/7] UNPACKING progress=35%
...
[7/7] READY template_id=tpl-abc123

Programmatic Go API

Embed the template creation logic directly in your Go application using the internal API:

package main

import (
	"context"
	"log"

	"github.com/tencentcloud/CubeSandbox/CubeMaster/pkg/templatecenter"
	"github.com/tencentcloud/CubeSandbox/CubeMaster/pkg/service/sandbox/types"
)

func main() {
	req := &types.CreateTemplateFromImageReq{
		Request:           &types.Request{RequestID: "my-request"},
		SourceImageRef:    "docker.io/library/nginx:1.25",
		WritableLayerSize: "10Gi",
		InstanceType:    "cubebox",
		NetworkType:     "tap",
	}

	downloadBaseURL := "http://master.example.com/artifacts"
	info, err := templatecenter.SubmitTemplateFromImage(context.Background(), req, downloadBaseURL)
	if err != nil {
		log.Fatalf("submit failed: %v", err)
	}
	log.Printf("template job submitted: job_id=%s template_id=%s", info.JobID, info.TemplateID)
}

This calls the same SubmitTemplateFromImage function used by the HTTP handler, triggering StreamRegistryToDir for the root filesystem extraction.

Debugging with Progress Callbacks

Inspect the native exporter's behavior by providing a progress callback when invoking StreamRegistryToDir directly:

// Inside CubeMaster/pkg/templatecenter/image/native.go context
func exampleProgress(bytes int) {
	log.Printf("downloaded %d bytes", bytes)
}

source := &image.PreparedSource{
    LocalRef:       "docker.io/library/busybox:latest",
    OnPullProgress: exampleProgress,
}

err := image.StreamRegistryToDir(context.Background(), source, "/tmp/busybox-rootfs")

The OnPullProgress callback receives incremental byte counts during the remote.Layer download and decompression phases, useful for debugging stuck pulls or bandwidth issues.

Core Source Files Reference

Understanding these files provides complete visibility into the conversion pipeline:

• CubeMaster/cmd/cubemastercli/commands/cubebox/template.go: Defines TemplateCreateFromImageCommand and TemplateWatchCommand for CLI interaction.
• CubeMaster/pkg/templatecenter/template_image.go: Implements SubmitTemplateFromImage for job orchestration and persistence.
• CubeMaster/pkg/templatecenter/image/native.go: Contains StreamRegistryToDir, the native OCI layer downloader and applier.
• CubeMaster/pkg/templatecenter/image/export.go: Routes to exportImageRootfs and selects between native, docker, and dockerless export modes.
• CubeMaster/pkg/templatecenter/image/types.go: Declares PreparedSource, ExportMode, and credential structures used throughout the exporter.

Summary

• CubeSandbox templates are created by converting OCI images into ext4 root filesystem artifacts through a master-coordinated job.
• The native exporter (StreamRegistryToDir in native.go) pulls layers concurrently using containerd/go-containerregistry, avoiding external CLI dependencies.
• Job lifecycle flows from TemplateCreateFromImageCommand CLI → SubmitTemplateFromImage master handler → runTemplateImageJob async runner → exportImageRootfs completion.
• Progress tracking occurs via progressReader streaming updates that the CLI displays through the template watch command.
• Resulting templates are stored in the RootfsArtifact table and referenced by template_id in subsequent sandbox create operations.

Frequently Asked Questions

What is the difference between native and Docker export modes in CubeSandbox?

The native mode uses the internal StreamRegistryToDir function that pulls OCI layers directly via Go libraries, while Docker mode shells out to the Docker CLI. Native mode is preferred because it eliminates external binary dependencies, supports concurrent layer downloads, and provides real-time progress callbacks without the overhead of Docker daemon communication.

How does CubeSandbox handle authentication for private OCI registries?

Authentication flows through the PreparedSource struct's credential fields. The CLI accepts --registry-username and --registry-password flags, which populate the remote.WithAuth option in StreamRegistryToDir. These credentials are passed securely to the containerd/go-containerregistry client when resolving the image manifest and pulling compressed layers.

Can I monitor the progress of a template creation job?

Yes. After submitting a template creation request, the master returns a job_id. Use the cubemastercli template watch --job-id <id> command or poll the /cube/template/from-image?job_id=<id> endpoint. The StreamRegistryToDir function reports progress through the progressReader, which updates the job status with current phase (PULLING, UNPACKING) and percentage completion.

What happens if a layer download fails during the export process?

The native exporter in StreamRegistryToDir downloads layers concurrently but applies them sequentially. If a layer fails to download or decompress, the job runner catches the error, cleans up temporary files from the destination parent directory, and marks the job status as FAILED with the error message. The CLI can then retry the operation with the same or modified parameters.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}