How to Convert OCI Images to Templates in CubeSandbox: A Complete Guide

CubeSandbox converts OCI images to templates by pulling image layers, extracting the root filesystem into an ext4 image via the native exporter, and storing the result as a template artifact through the cubemastercli template create-from-image command.

The TencentCloud/CubeSandbox repository provides a native OCI-to-template conversion pipeline that eliminates external dependencies like Docker or Skopeo. This process extracts container layers directly from registry storage, streams them into a sandbox-ready root filesystem, and packages the result for rapid deployment.

Understanding the OCI to Template Conversion Workflow

The conversion process operates across three architectural layers: the CLI client, the master HTTP service, and the job runner that executes the native export.

CLI Entry Point and Request Assembly

The conversion begins in CubeMaster/cmd/cubemastercli/commands/cubebox/template.go within the TemplateCreateFromImageCommand function (lines 751–770). This handler parses user flags—such as --image, --writable-layer-size, --network-type, and --allow-internet-access—and constructs a types.CreateTemplateFromImageReq request.

The CLI validates registry credentials, merges network configuration overrides, and POSTs the assembled request to the master endpoint /cube/template/from-image. The master returns a TemplateImageJobInfo containing a unique job_id that tracks the asynchronous conversion process.

Master Service Submission and Job Creation

Upon receiving the request, the master invokes SubmitTemplateFromImage in CubeMaster/pkg/templatecenter/template_image.go (lines 95–105). This function normalizes the request, checks for duplicate jobs, creates a persistent TemplateImageJob record, and spawns runTemplateImageJob in a dedicated goroutine.

The job runner manages the lifecycle of the conversion, handling retries, progress reporting, and eventual artifact storage. It delegates the actual filesystem extraction to the export subsystem based on the request’s ExportMode.

Native Root Filesystem Export

When ExportMode is set to native (the default), the job runner calls exportImageRootfs, which delegates to StreamRegistryToDir in CubeMaster/pkg/templatecenter/image/native.go (lines 74–78). This implementation:

1. Resolves the OCI reference via nativeImageForSource to determine the manifest and layer digests.
2. Pulls layers concurrently using containerd/go-containerregistry with remote.WithAuth and layer.Compressed, respecting the nativeExportConcurrency limit.
3. Streams and decompresses each layer through compression.DecompressStream and applies it directly to the destination directory using archive.Apply.
4. Cleans up temporary files immediately after application to minimize disk footprint, writing temp files under the destination’s parent directory to ensure atomic renames avoid EXDEV errors.

Step-by-Step Implementation in the Source Code

The following breakdown maps the user-facing command to specific source code locations:

Step	Component	Function	Purpose
1	CLI	TemplateCreateFromImageCommand	Parses flags and builds CreateTemplateFromImageReq
2	HTTP Client	doHttpReq	POSTs to /cube/template/from-image, receives job_id
3	Master Handler	SubmitTemplateFromImage	Creates job record and spawns background worker
4	Job Runner	runTemplateImageJob → exportImageRootfs	Orchestrates the export process
5	Native Exporter	StreamRegistryToDir	Downloads and unpacks layers concurrently
6	Artifact Handler	(internal)	Creates ext4 image and updates RootfsArtifact table

After step 6, the template status transitions to READY, making it available for sandbox create operations.

Practical Code Examples

CLI Usage with Common Flags

The most common method to convert OCI images to templates uses the cubemastercli binary:

# Initiate conversion and watch until completion

cubemastercli template create-from-image \
    --image docker.io/library/ubuntu:22.04 \
    --writable-layer-size 20Gi \
    --instance-type cubebox \
    --network-type tap \
    --node 10.0.1.5 \
    --allow-internet-access \
    --expose-port 80 \
    --registry-username myuser \
    --registry-password mypass

This command triggers the full pipeline: the CLI polls the master job status via /cube/template/from-image?job_id=... and displays progress steps such as [1/7] PULLING progress=12% and [2/7] UNPACKING progress=35%.

Programmatic API Integration

Embed the conversion logic directly in Go applications using the internal API:

package main

import (
	"context"
	"log"

	"github.com/tencentcloud/CubeSandbox/CubeMaster/pkg/templatecenter"
	"github.com/tencentcloud/CubeSandbox/CubeMaster/pkg/service/sandbox/types"
)

func main() {
	req := &types.CreateTemplateFromImageReq{
		Request:           &types.Request{RequestID: "my-request"},
		SourceImageRef:    "docker.io/library/nginx:1.25",
		WritableLayerSize: "10Gi",
		InstanceType:      "cubebox",
		NetworkType:       "tap",
	}
	
	// Base URL where the master serves artifacts
	downloadBaseURL := "http://master.example.com/artifacts"

	info, err := templatecenter.SubmitTemplateFromImage(context.Background(), req, downloadBaseURL)
	if err != nil {
		log.Fatalf("submit failed: %v", err)
	}
	log.Printf("template job submitted: job_id=%s template_id=%s", info.JobID, info.TemplateID)
}

This calls the same SubmitTemplateFromImage function used by the HTTP handler, initiating the StreamRegistryToDir workflow internally.

Debugging with Progress Callbacks

Monitor layer download progress by implementing the callback interface defined in CubeMaster/pkg/templatecenter/image/types.go:

// Example progress handler
func progressHandler(bytesDownloaded int64) {
	log.Printf("downloaded %d bytes", bytesDownloaded)
}

// Usage with the native exporter
source := &image.PreparedSource{
    LocalRef:       "docker.io/library/busybox:latest",
    OnPullProgress: progressHandler,
}

err := image.StreamRegistryToDir(context.Background(), source, "/tmp/busybox-rootfs")
if err != nil {
    log.Fatalf("export failed: %v", err)
}

The progressReader reports incremental byte counts, layer completion, and network throughput, enabling real-time UI updates during long-running pulls.

Why the Native Exporter is Preferred

CubeSandbox implements three export modes (native, dockerless, and docker), but the native path is recommended for production deployments:

• Zero external dependencies: Avoids requiring Docker, Skopeo, or Umoci binaries on the master node.
• Concurrent prefetching: Uses nativeExportConcurrency to download multiple layers simultaneously, maximizing network utilization.
• Streaming decompression: Layers are decompressed and applied on-the-fly without intermediate storage, reducing I/O overhead.
• Safe temp file handling: Temporary layer files are written to the same filesystem as the destination, ensuring atomic os.Rename operations prevent EXDEV cross-device errors.

Summary

• CubeSandbox converts OCI images to templates through the cubemastercli template create-from-image command, which triggers an asynchronous job pipeline.
• The native exporter (StreamRegistryToDir in native.go) pulls layers concurrently using containerd/go-containerregistry, decompresses them via compression.DecompressStream, and applies them with archive.Apply.
• Key source files include template.go for CLI handling, template_image.go for job orchestration, and native.go for the core OCI extraction logic.
• Temporary file cleanup happens immediately after each layer is applied, keeping disk usage minimal during the conversion process.
• Progress tracking is available through the OnPullProgress callback and CLI watch commands that poll the master job status.

Frequently Asked Questions

What is the difference between native and docker export modes in CubeSandbox?

The native mode uses the internal StreamRegistryToDir function to pull and unpack OCI layers directly without external binaries, while docker mode shells out to the Docker CLI and dockerless mode relies on Skopeo or Umoci. Native mode is preferred because it eliminates dependency management, reduces attack surface, and provides better performance through concurrent layer fetching and streaming decompression.

How do I monitor the progress of an OCI image conversion?

Use the cubemastercli template watch --job-id <id> command or poll cubemastercli template status --job-id <id>. The master reports granular progress stages including PULLING, UNPACKING, and READY. For programmatic access, the PreparedSource struct accepts an OnPullProgress callback that receives byte counts during the StreamRegistryToDir execution.

Can I convert private registry images requiring authentication?

Yes. Pass --registry-username and --registry-password flags to the CLI, or populate the Auth field in CreateTemplateFromImageReq. The native exporter passes these credentials to containerd/go-containerregistry via remote.WithAuth, supporting standard Docker registry authentication for private repositories.

What happens to temporary files during the conversion process?

The native exporter writes temporary layer files underneath the destination directory's parent path. After archive.Apply successfully unpacks a layer, the temporary file is deleted immediately. This design ensures that the final os.Rename operation never crosses filesystem boundaries (preventing EXDEV errors) and keeps disk utilization low by cleaning up each layer before downloading the next batch.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}