What is Nydus? Understanding the Container Image Service and RAFS Architecture

Nydus is a container-image service developed by the Dragonfly OSS community that introduces the RAFS (Remote Artifact File System) format and a userspace daemon to enable on-demand, chunk-level image fetching, eliminating the need to download full layers before container startup.

Nydus is an open-source project hosted at dragonflyoss/nydus that reimagines how container images are stored, distributed, and consumed. Unlike traditional OCI images that require pulling complete layers before a container can execute, Nydus implements a lazy-loading architecture that fetches only the necessary data chunks on demand. This approach dramatically reduces cold-start latency, bandwidth consumption, and storage redundancy for modern cloud-native workloads.

Core Architecture of Nydus

The Nydus architecture decouples image metadata from data storage, enabling efficient remote access. The system comprises three primary components: the RAFS image format, the nydusd daemon, and the backend adapter layer.

RAFS Image Format

The RAFS (Remote Artifact File System) format stores container images as content-addressable artifacts consisting of a bootstrap file (metadata) and blob files (data). According to docs/nydus-design.md, the bootstrap contains a merkle-tree of inodes where each inode points to 1 MiB chunks within the blobs. This design allows nydusd to fetch only the specific chunks required for a container's initial process, rather than entire layers.

The format supports chunk-level deduplication, ensuring identical chunks across different images or layers are stored only once. The bootstrap records cryptographic digests (SHA-256 or BLAKE3) for each chunk, enabling runtime integrity verification.

Nydusd Daemon

The nydusd userspace daemon, implemented in src/lib.rs, serves the RAFS image to the kernel via FUSE or virtiofs. When a container starts, nydusd reads the bootstrap on-demand, validates metadata and chunk digests, and pulls missing chunks from the configured backend. The daemon maintains a local cache to avoid redundant remote fetches.

The daemon handles integrity verification by validating each chunk's digest on read, aborting with EINVAL if corruption is detected. This ensures that data fetched from remote storage backends is trustworthy before being served to the container.

Backend Adapters and Nydusify CLI

Backend adapters provide pluggable storage access to various remote services. The contrib/nydusify/pkg/backend directory contains implementations for OCI registries, Alibaba Cloud OSS, AWS S3, local filesystem, and Dragonfly P2P networks.

The nydusify CLI tool, defined in contrib/nydusify/cmd/nydusify.go, orchestrates the conversion workflow. It pulls OCI images, invokes nydus-image to construct RAFS artifacts, and pushes results to the target backend. The tool also provides sub-commands for checking image integrity, mounting RAFS images locally, optimizing chunk layouts, and copying images between registries.

Problems Nydus Solves for Container Workloads

Nydus addresses fundamental inefficiencies in traditional container image distribution through its lazy-loading architecture and content-addressable storage model.

Slow Container Startup Times

Traditional OCI images require downloading complete layers before the container process can execute. For large images, this creates significant cold-start latency. Nydus solves this by splitting the rootfs into 1 MiB chunks and streaming only the chunks required for the initial process startup. In production environments, this often reduces startup time from minutes to seconds by fetching less than 5% of the total image size.

Redundant Data Transfer and Storage

Identical files across image layers or different images are repeatedly transmitted and stored in traditional registries. Nydus implements chunk-level deduplication, ensuring each unique 1 MiB chunk is stored once and referenced by multiple inodes across different images. The bootstrap records only chunk digests, eliminating storage waste and reducing transfer volumes.

Excessive Network Bandwidth Consumption

Downloading full layers wastes bandwidth, particularly in large Kubernetes clusters where identical images are pulled to thousands of nodes. Nydus enables on-demand loading that pulls only the fraction of the image actually accessed during container runtime. The daemon can also utilize pre-fetch hints to mitigate latency spikes for predictable access patterns, further reducing network pressure.

Lack of Runtime Integrity Guarantees

Corrupted image layers can cause silent failures or security vulnerabilities in production containers. Nydus provides cryptographic integrity verification at both the metadata and data levels. The bootstrap contains a merkle-tree of inodes, and each data chunk includes SHA-256 or BLAKE3 digests. The nydusd daemon validates each chunk on read, aborting with EINVAL if corruption is detected.

Inflexible Storage Backend Requirements

Standard OCI tooling assumes registry storage, making it difficult to use alternative storage services. Nydus defines a backend abstraction layer with implementations for OCI registries, Alibaba Cloud OSS, AWS S3, local filesystem, and Dragonfly P2P networks. This flexibility allows organizations to leverage existing storage infrastructure without modifying the core image format.

Runtime Compatibility Constraints

Existing container runtimes expect a POSIX overlayfs layout, making it difficult to adopt alternative image formats. Nydus maintains OCI compatibility by serving RAFS images via FUSE (through nydusd) or in-kernel EROFS (RAFS v6), presenting a standard root filesystem to containers. The project provides snapshotter plugins for containerd and integration with Kubernetes CRI, nerdctl, and Kata Containers.

Practical Nydus Workflows and Code Examples

The following command-line snippets demonstrate typical Nydus operations using the nydusify and nydus-image binaries available on the project's Release page.

Convert an OCI Image to Nydus Format

# Pull an image from a registry and convert it to Nydus RAFS format

nydusify convert \
  --source docker.io/library/alpine:3.18 \
  --target myregistry.example.com/nydus/alpine:3.18-nydus

This command fetches the source image, invokes nydus-image create to build a RAFS bootstrap and blob files, and pushes the resulting artifacts to the target registry or configured backend.

Build a RAFS Filesystem from a Local Directory

nydus-image create -t dir-rafs \
  -D /tmp/nydus-output \
  /path/to/app/rootfs

The output directory contains a bootstrap file (metadata) and one or more blob files (chunked data), ready for upload to remote storage.

Run a Container with Nydus Image

# Using nerdctl with the nydus snapshotter

nerdctl --snapshotter nydus run -d \
  myregistry.example.com/nydus/alpine:3.18-nydus \
  sleep 3600

The nydus-snapshotter automatically launches nydusd, which lazily pulls only the chunks required for the sleep binary, achieving sub-second startup times for large images.

Inspect RAFS Metadata

nydus-image inspect /tmp/nydus-output/bootstrap

This displays the merkle-tree layout, chunk digests, compression algorithms (e.g., COMPRESS_ZSTD), and enabled features (e.g., HASH_BLAKE3).

Compact and Optimize Existing Images

nydus-image compact \
  --bootstrap /tmp/nydus-output/bootstrap \
  --backend-type oss \
  --backend-config-file backend.json \
  --config compact.json

The compact operation removes unused chunks, merges small blobs, and can apply a chunk-dictionary for cross-image deduplication, reducing storage footprint.

Summary

• Nydus is a Dragonfly OSS container-image service that replaces traditional layer-based image distribution with a chunk-level, on-demand fetching system.
• The RAFS format splits images into metadata (bootstrap) and data (blob) components, enabling 1 MiB chunk granularity with cryptographic integrity verification.
• Nydusd serves RAFS images via FUSE or virtiofs, validating chunk digests on read and aborting on corruption with EINVAL.
• Nydusify provides a high-level CLI for converting OCI images, with backend adapters supporting registries, S3, OSS, and Dragonfly P2P networks.
• Integration with containerd, Kubernetes CRI, nerdctl, and Kata Containers allows Nydus to replace standard overlayfs without modifying container runtimes.

Frequently Asked Questions

What is the difference between Nydus and standard OCI images?

Standard OCI images are organized as tar-gzipped layers that must be downloaded and extracted completely before a container can start. Nydus uses the RAFS format to separate image metadata from data, organizing content into 1 MiB chunks that are fetched on-demand via nydusd. This architecture reduces startup time from minutes to seconds for large images and enables chunk-level deduplication across different images.

How does Nydus ensure data integrity during lazy loading?

Nydus implements cryptographic verification at multiple levels. The bootstrap metadata file contains a merkle-tree of inodes where each node references chunk digests using SHA-256 or BLAKE3. When nydusd serves a file via FUSE, it validates each chunk's digest against the expected hash stored in the bootstrap. If corruption is detected, the daemon returns EINVAL and aborts the operation, preventing contaminated data from reaching the container.

Can Nydus work with existing container runtimes like containerd and Kubernetes?

Yes, Nydus integrates seamlessly with existing ecosystems through the nydus-snapshotter plugin for containerd. This snapshotter communicates with nydusd to mount RAFS images as container root filesystems. The integration extends to Kubernetes via CRI support, allowing pods to specify Nydus images in their manifests. Additionally, tools like nerdctl and Kata Containers support Nydus through the standard snapshotter interface, requiring no changes to the container runtime itself.

What storage backends are supported by Nydus?

Nydus abstracts storage access through a backend adapter layer defined in contrib/nydusify/pkg/backend. Supported backends include OCI-compliant registries (Docker Hub, Harbor, etc.), Alibaba Cloud OSS, AWS S3, local filesystem, and Dragonfly P2P networks. This flexibility allows organizations to deploy Nydus images on existing storage infrastructure without vendor lock-in, while the nydusify CLI handles authentication and transport configuration for each backend type.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}