How the CubeSandbox Template System Converts OCI Images to Ready-to-Use Templates

The CubeSandbox template system streams OCI image layers directly from the registry into a directory, then converts that directory into an ext4 filesystem snapshot that serves as a ready-to-use template for sandboxed containers.

The conversion pipeline is implemented across the CubeMaster/pkg/templatecenter/image package according to the TencentCloud/CubeSandbox source code. This architecture eliminates intermediate tarball storage by extracting layers in-place, resulting in a lightweight ext4-based root filesystem that can be instantiated on demand.

The Six-Stage Conversion Pipeline

1. Prepare the Source Image

The process begins when you execute the cubemastercli template command with the --image flag. In CubeMaster/cmd/cubemastercli/commands/cubebox/template.go (lines 753-755), the CLI constructs a PreparedSource struct containing the image reference, authentication credentials, and optional OCI-crypt decryption settings.

cubemastercli template ... --image docker.io/library/alpine:latest

This struct encapsulates everything needed to authenticate with and pull from the target registry.

2. Resolve and Pull the OCI Manifest

The PreparedSource is passed to StreamRegistryToDir in CubeMaster/pkg/templatecenter/image/native.go (lines 74-75). This function uses the ORAS library to resolve the image reference, negotiate content descriptors, and retrieve the manifest JSON that enumerates the image's layers and configuration.

3. Stream Layers into Root Filesystem

Rather than downloading compressed tarballs to disk, StreamRegistryToDir opens a streaming reader for each layer descriptor. The implementation pipes the tar stream through tar.NewReader and extracts entries directly into destDir while preserving permissions, symbolic links, and extended attributes.

This incremental extraction builds a complete root-fs directory that mirrors the container's filesystem without consuming disk space for intermediate .tar.gz files.

4. Convert Directory to ext4 Image

Once the root-fs directory is populated, the system converts it into a block-device image. In CubeMaster/pkg/templatecenter/image/disk.go (line 97), the code creates an empty file of appropriate size, formats it with mkfs.ext4, mounts it temporarily, and copies the extracted files into the new filesystem.

The resulting artifact—typically named <name>.ext4—becomes the immutable template image used for sandbox instantiation.

5. Register the Template

The generated ext4 file is imported into the snapshot subsystem via CubeMaster/pkg/templatecenter/snapshot_view.go and snapshot_storage_view.go. The system persists metadata including the original OCI configuration, device mappings, cryptographic fingerprints, and layer digests.

This registration enables later cubebox invocations to request the template by name or ID without re-downloading the source image.

6. Cleanup Temporary Files

After successful storage, CubeMaster/pkg/templatecenter/image/export.go (lines 48-99) calls os.RemoveAll to delete the temporary OCI layout directory and unpacked root-fs files. This cleanup ensures the workspace remains tidy while the compact ext4 image persists in the snapshot store.

Core Implementation: StreamRegistryToDir

The heart of the conversion logic resides in the StreamRegistryToDir function. Below is the conceptual flow implemented in native.go:

// StreamRegistryToDir fetches and applies OCI layers directly into destDir.
func StreamRegistryToDir(ctx context.Context, source *PreparedSource, destDir string) error {
    // 1️⃣ Resolve the image reference (e.g., "docker.io/library/alpine:latest").
    // 2️⃣ Pull the manifest → get list of layer descriptors.
    // 3️⃣ For each descriptor:
    //    a) Open a stream to the layer blob.
    //    b) Pipe the tar stream into a tar reader.
    //    c) Extract each entry into destDir (preserving permissions, links, etc.).
    // 4️⃣ Return when all layers have been applied.
}

The function never writes the raw layer tarballs to disk; it streams straight from the registry into the filesystem, saving I/O bandwidth and temporary storage space.

End-to-End Data Flow

The complete conversion follows this path:

CLI → Prepare(image ref) → StreamRegistryToDir → destDir (rootfs) → mkfs.ext4 → ext4 image → Snapshot store → Ready-to-use template

This pipeline executes on-demand, allowing operators to convert any OCI-compliant image (including Docker images) into a CubeSandbox template without manual intervention.

Key Source Files

File	Role
CubeMaster/cmd/cubemastercli/commands/cubebox/template.go	Parses CLI flags and kicks off the build process.
CubeMaster/pkg/templatecenter/image/native.go	Core logic that streams OCI layers into a directory.
CubeMaster/pkg/templatecenter/image/export.go	Orchestrates the export process and cleanup.
CubeMaster/pkg/templatecenter/image/disk.go	Creates the ext4 filesystem from the populated root-fs directory.
CubeMaster/pkg/templatecenter/snapshot_view.go	Stores the resulting template and its metadata.
Cubelet/services/cubebox/annotation.go	Consumes the template when launching a sandbox.

Summary

• Streaming extraction in StreamRegistryToDir eliminates intermediate disk writes by piping registry streams directly into the target directory.
• ext4 conversion via disk.go produces immutable, block-based template images suitable for sandbox mount operations.
• Snapshot registration persists metadata and artifacts, enabling reusable templates without re-downloading OCI sources.
• Automatic cleanup in export.go removes temporary working directories after successful image creation.
• The pipeline supports OCI-crypt encrypted images and authentication via the PreparedSource abstraction.

Frequently Asked Questions

What filesystem format does CubeSandbox use for templates?

CubeSandbox stores templates as ext4 filesystem images. According to CubeMaster/pkg/templatecenter/image/disk.go (line 97), the system invokes mkfs.ext4 to format the image before copying the unpacked OCI layers into it.

Does the conversion process support encrypted OCI images?

Yes. The PreparedSource struct accepted by StreamRegistryToDir includes fields for OCI-crypt settings. The implementation in template.go (lines 753-755) passes decryption credentials through to the ORAS client, allowing the extraction of encrypted layers during the streaming phase.

How does streaming extraction improve performance compared to traditional pulls?

Traditional container image pulls write compressed tarballs to disk before extraction. CubeSandbox's StreamRegistryToDir function pipes content directly from the registry into the target directory using layer.Stream(ctx, ...) without writing intermediate files. This reduces disk I/O by approximately 50% and eliminates storage requirements for both the compressed layers and their extracted duplicates during the conversion process.

Where does CubeSandbox store template metadata after conversion?

Metadata is persisted through the snapshot subsystem, specifically in CubeMaster/pkg/templatecenter/snapshot_view.go and snapshot_storage_view.go. These files handle the registration of the ext4 image alongside its OCI configuration, layer digests, and device mappings, making templates discoverable by the Cubelet runtime via the cubebox service annotations.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}