How to Convert OCI Images to CubeSandbox Templates: A Complete Guide

CubeSandbox converts OCI images to templates by streaming layers from a registry, decompressing them on-the-fly, and packaging the resulting root filesystem into an ext4 image using the native exporter in StreamRegistryToDir.

The TencentCloud/CubeSandbox repository provides a complete template system that transforms standard OCI container images into sandbox-ready templates without requiring Docker or other external tools. This conversion process extracts the root filesystem, creates a writable ext4 layer, and registers the artifact for subsequent sandbox creation. Understanding how to convert OCI images to CubeSandbox templates enables you to build custom sandbox environments from any public or private container registry.

The Three-Layer Conversion Workflow

The conversion process operates through three distinct layers: client command parsing, master service coordination, and asynchronous root filesystem export.

CLI Entry Point (TemplateCreateFromImageCommand)

The journey begins in CubeMaster/cmd/cubemastercli/commands/cubebox/template.go where the TemplateCreateFromImageCommand function parses user-provided flags. This command accepts parameters including --image for the OCI reference, --writable-layer-size for the overlay size, --node for target placement, and --allow-internet-access for network configuration.

The CLI assembles these inputs into a types.CreateTemplateFromImageReq struct and submits it via HTTP POST to the master's /cube/template/from-image endpoint. Upon submission, the master returns a job_id that the client uses to track progress through subsequent polling or watch commands.

Master Service Submission (SubmitTemplateFromImage)

On the server side, CubeMaster/pkg/templatecenter/template_image.go houses the SubmitTemplateFromImage function. This handler normalizes the incoming request, checks for existing duplicate jobs, creates a persistent TemplateImageJob record, and spawns runTemplateImageJob in a background goroutine.

The master immediately returns a TemplateImageJobInfo response containing the job_id and template_id, allowing the client to monitor the asynchronous operation without blocking the connection.

Root Filesystem Export (StreamRegistryToDir)

The actual heavy lifting occurs in CubeMaster/pkg/templatecenter/image/native.go within the StreamRegistryToDir function. When the request's ExportMode is set to native (the default), the job runner calls exportImageRootfs, which delegates to this native streamer.

The process resolves the OCI reference via nativeImageForSource, then pulls each layer concurrently using containerd/go-containerregistry. It streams compressed data through a progressReader, decompresses layers on-the-fly using compression.DecompressStream, and applies them directly to the destination directory via archive.Apply. Temporary files are deleted immediately after application to minimize disk footprint.

Deep Dive into the Native Export Process

The native exporter offers significant advantages over legacy Docker-based alternatives. It eliminates external binary dependencies, implements concurrent prefetching for maximum network throughput, and provides low-latency progress reporting through the progressReader callback mechanism.

Safety considerations include writing temporary files under the destination's parent directory, ensuring that the final os.Rename operation occurs on the same filesystem and avoids EXDEV cross-device errors.

Step-by-Step Implementation Flow

1. Flag Parsing – The CLI command cubemastercli template create-from-image validates inputs and builds the request object in template.go.

2. HTTP Submission – The client sends the request to POST /cube/template/from-image and receives a job_id for tracking.

3. Job Initialization – SubmitTemplateFromImage validates the request, checks for existing jobs, and persists a new job record before launching the async runner.

4. Layer Streaming – The job runner executes exportImageRootfs → StreamRegistryToDir, which downloads layers in parallel using nativeExportConcurrency settings.

5. Artifact Creation – After the root filesystem directory is populated, the system creates an ext4 image using mkfs.ext4, stores it in the RootfsArtifact table, and updates the job status to READY.

6. Client Monitoring – Users poll via template watch --job-id <id> or check template status to retrieve the final template ID for sandbox creation.

Command Line Examples

The following command pulls an Ubuntu image and creates a template with a 20GB writable layer:

cubemastercli template create-from-image \
    --image docker.io/library/ubuntu:22.04 \
    --writable-layer-size 20Gi \
    --instance-type cubebox \
    --network-type tap \
    --node 10.0.1.5 \
    --allow-internet-access \
    --expose-port 80 \
    --registry-username myuser \
    --registry-password mypass

The CLI automatically enters watch mode, displaying progress steps:

[1/7] PULLING progress=12%
[2/7] UNPACKING progress=35%
...

Programmatic Usage

Embed the same conversion logic directly in Go applications using the internal API:

package main

import (
	"context"
	"log"

	"github.com/tencentcloud/CubeSandbox/CubeMaster/pkg/templatecenter"
	"github.com/tencentcloud/CubeSandbox/CubeMaster/pkg/service/sandbox/types"
)

func main() {
	req := &types.CreateTemplateFromImageReq{
		Request:        &types.Request{RequestID: "my-request"},
		SourceImageRef: "docker.io/library/nginx:1.25",
		WritableLayerSize: "10Gi",
		InstanceType:   "cubebox",
		NetworkType:    "tap",
	}
	downloadBaseURL := "http://master.example.com/artifacts"

	info, err := templatecenter.SubmitTemplateFromImage(context.Background(), req, downloadBaseURL)
	if err != nil {
		log.Fatalf("submit failed: %v", err)
	}
	log.Printf("template job submitted: job_id=%s template_id=%s", info.JobID, info.TemplateID)
}

For debugging the export process, inspect the progress callback mechanism:

// Inside native.go – progress callback example
func exampleProgress(bytes int) {
	log.Printf("downloaded %d bytes", bytes)
}

source := &image.PreparedSource{
    LocalRef: "docker.io/library/busybox:latest",
    OnPullProgress: exampleProgress,
}
err := image.StreamRegistryToDir(context.Background(), source, "/tmp/busybox-rootfs")

Key Source Files and Functions

• CubeMaster/cmd/cubemastercli/commands/cubebox/template.go – Contains TemplateCreateFromImageCommand for CLI flag parsing and request assembly.
• CubeMaster/pkg/templatecenter/template_image.go – Houses SubmitTemplateFromImage for master-side job orchestration and persistence.
• CubeMaster/pkg/templatecenter/image/native.go – Implements StreamRegistryToDir, the core native OCI exporter that handles concurrent layer streaming and on-the-fly decompression.
• CubeMaster/pkg/templatecenter/image/export.go – Contains exportImageRootfs which selects between native, dockerless, and docker export modes.
• CubeMaster/pkg/templatecenter/image/types.go – Defines data structures including PreparedSource, ExportMode, and credential handling used throughout the export pipeline.

Summary

• CubeSandbox converts OCI images to templates through a three-stage pipeline: CLI parsing, master submission, and async root filesystem export.
• The native exporter in StreamRegistryToDir eliminates Docker dependencies by directly streaming and decompressing OCI layers using containerd/go-containerregistry.
• Concurrent layer fetching with immediate cleanup minimizes disk usage and maximizes network throughput during the conversion process.
• Job tracking via job_id allows asynchronous monitoring of the template creation process from CLI or programmatic interfaces.

Frequently Asked Questions

What is the difference between native mode and docker export mode?

Native mode uses the internal StreamRegistryToDir function to pull OCI layers directly without external dependencies, while docker export mode relies on the Docker CLI or daemon to extract the root filesystem. Native mode is preferred because it offers better performance through concurrent downloads, lower resource overhead, and eliminates the need for Docker installation on the master node.

How does the system handle authentication for private registries?

The CreateTemplateFromImageReq accepts RegistryUsername and RegistryPassword fields that propagate through to remote.WithAuth in the containerd/go-containerregistry client. These credentials are used when resolving the image reference and pulling layers, ensuring secure access to private OCI repositories without storing credentials in the final template artifact.

Can I monitor the conversion progress in real-time?

Yes, the cubemastercli template watch --job-id <id> command polls the master's status endpoint and displays incremental progress including pull percentages, unpacking status, and layer completion. Internally, the progressReader wrapper reports byte counts to the OnPullProgress callback, which feeds into the job status updates visible to the CLI.

What happens if the conversion job fails midway?

The master service maintains persistent job records in the TemplateImageJob table, allowing you to inspect failure reasons via cubemastercli template status. Temporary layer files are written to the destination parent directory and cleaned up immediately after application or on job cancellation, preventing disk space leakage from incomplete conversions.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}