Nydus POSIX Compatibility via EROFS and FUSE: Implementation Guide

Nydus provides full POSIX-compatible container filesystem access through either in-kernel EROFS (for RAFS v6 images) or user-space FUSE (for RAFS v5 images), combined with overlayfs for writable layers.

The dragonflyoss/nydus repository implements a container image acceleration service that exposes Nydus images as standard POSIX filesystems. By leveraging either the EROFS driver for kernel-native performance or FUSE for user-space flexibility, Nydus ensures that container runtimes interact with accelerated images exactly as they would with traditional overlay filesystems.

How Nydus Implements POSIX Compatibility

Nydus achieves POSIX compatibility through a dual-path architecture that supports two distinct RAFS (Registry Acceleration File System) format versions. The RAFS v6 format enables direct kernel mounting via EROFS, while RAFS v5 relies on the nydusd FUSE server to intercept and serve filesystem calls.

Both paths ultimately present a read-only lower layer that, when combined with an overlayfs upper directory, provides a fully writable POSIX-compliant filesystem stack. This design ensures that standard operations like chmod, rename, and utimes function correctly through the overlay mechanism.

EROFS Path: In-Kernel POSIX Support

RAFS v6 Format and EROFS Driver

When Nydus images are built using the RAFS v6 format, the metadata structure aligns with the Linux EROFS (Enhanced Read-Only File System) on-disk layout. As defined in rafs/src/metadata/layout/v6.rs, this format allows the kernel EROFS driver to mount the image directly without user-space intervention for read operations.

The kernel EROFS driver implements standard POSIX VFS operations, meaning all file system calls—open, read, stat, readdir—execute through native kernel code paths. This provides optimal performance and full compatibility with standard Unix semantics.

Block Device Implementation

To expose RAFS v6 images to the kernel, nydus-service creates a virtual block device. The implementation in service/src/block_device.rs registers a /dev/nydusX device node that presents the Nydus image as a standard block storage device.

Once the block device is available, system administrators mount it using the standard EROFS mount command:

sudo mount -t erofs /dev/nydus0 /mnt/nydus-erofs

This mount point then serves as the lower directory in an overlayfs stack, providing the foundation for a writable POSIX filesystem.

FUSE Path: User-Space POSIX Support

FUSE Server Implementation

For RAFS v5 images, Nydus implements a FUSE (Filesystem in Userspace) server within the nydusd daemon. The implementation in service/src/fusedev.rs opens /dev/fuse and registers filesystem callbacks that translate kernel VFS requests into Nydus storage backend operations.

When a process accesses a file within the FUSE mount point, the kernel routes the request to nydusd, which retrieves the data from the Nydus image blobs (stored in registry, OSS, or local filesystem). The daemon then returns the data to the kernel, which presents it to the calling process as standard POSIX file content.

Integration with Overlayfs

To provide full POSIX write semantics, the FUSE mount (read-only) combines with an overlayfs upper layer. As documented in docs/nydusd.md, users create a writable overlay structure:

sudo mkdir -p /var/lib/nydus/upper /var/lib/nydus/work
sudo mount -t overlay overlay \
  -o lowerdir=/var/lib/nydus/fuse-mnt,upperdir=/var/lib/nydus/upper,workdir=/var/lib/nydus/work \
  /var/lib/nydus/posix-root

The resulting /var/lib/nydus/posix-root supports all standard POSIX operations. Write operations modify the overlayfs upper directory, while read operations fetch data from the Nydus FUSE mount, creating a seamless POSIX-compatible container root filesystem.

Architectural Comparison

Feature	EROFS Path (RAFS v6)	FUSE Path (RAFS v5)
Kernel Integration	Native EROFS driver (fs/erofs)	FUSE module (/dev/fuse)
Performance	Zero-copy kernel paths; minimal overhead	User-space context switches; higher latency
Daemon Role	Block device provider (block_device.rs)	Full filesystem server (fusedev.rs)
Mount Command	mount -t erofs /dev/nydus0 …	nydusd --mountpoint … then overlayfs
POSIX Compliance	Full via kernel VFS	Full via FUSE + overlayfs

The service/src/lib.rs module explicitly lists both fuse and erofs as supported services, confirming that Nydus maintains POSIX compatibility across both implementation paths.

Practical Implementation: Code Examples

Mounting with FUSE and Overlayfs

To deploy Nydus with FUSE-based POSIX support, configure the daemon and overlay structure:

# Start the FUSE daemon

sudo nydusd \
  --config /etc/nydus/nydusd-config.localfs.json \
  --mountpoint /var/lib/nydus/fuse-mnt \
  --bootstrap /path/to/bootstrap \
  --log-level info

# Create overlayfs for write support

sudo mkdir -p /var/lib/nydus/upper /var/lib/nydus/work
sudo mount -t overlay overlay \
  -o lowerdir=/var/lib/nydus/fuse-mnt,upperdir=/var/lib/nydus/upper,workdir=/var/lib/nydus/work \
  /var/lib/nydus/posix-root

Mounting with EROFS Block Device

For kernel-native POSIX access using EROFS:

# Build RAFS v6 image first, then start daemon in block device mode

sudo nydusd \
  --config /etc/nydus/nydusd-config.localfs.json \
  --blockdev /dev/nydus0 \
  --bootstrap /path/to/bootstrap \
  --log-level info

# Mount via kernel EROFS driver

sudo mkdir -p /mnt/nydus-erofs
sudo mount -t erofs /dev/nydus0 /mnt/nydus-erofs

# Add overlayfs for write support (same pattern as FUSE)

sudo mkdir -p /mnt/nydus-erofs/upper /mnt/nydus-erofs/work
sudo mount -t overlay overlay \
  -o lowerdir=/mnt/nydus-erofs,upperdir=/mnt/nydus-erofs/upper,workdir=/mnt/nydus-erofs/work \
  /mnt/nydus-posix

The docs/nydus-fscache.md document provides additional details on configuring the fscache-based EROFS path for high-performance container workloads.

Summary

• Nydus delivers full POSIX compatibility for container images through dual backend implementations.
• EROFS path: Uses RAFS v6 format and service/src/block_device.rs to expose virtual block devices mounted by the kernel EROFS driver, providing native POSIX performance.
• FUSE path: Uses RAFS v5 format and service/src/fusedev.rs to serve filesystem requests via /dev/fuse, combined with overlayfs for write support.
• Both paths integrate with overlayfs to provide writable POSIX semantics required by container runtimes.
• The architecture is explicitly documented in service/src/lib.rs and the top-level README.md, confirming production-ready POSIX support.

Frequently Asked Questions

Does Nydus support standard POSIX file operations like chmod and rename?

Yes. When Nydus is mounted with an overlayfs upper layer—whether using the EROFS or FUSE backend—all standard POSIX operations function normally. The overlayfs handles mutable operations like chmod, rename, and utimes by modifying the writable upper directory, while the Nydus layer provides the immutable base filesystem. This design ensures container runtimes experience standard Unix semantics without modification.

What is the difference between RAFS v5 and RAFS v6 in terms of POSIX support?

Both formats provide equivalent POSIX compatibility but through different kernel mechanisms. RAFS v6 aligns with the EROFS on-disk format defined in rafs/src/metadata/layout/v6.rs, allowing the kernel EROFS driver to mount the image directly as a block device. RAFS v5 requires the nydusd FUSE server to handle filesystem calls via /dev/fuse. While both support full POSIX when combined with overlayfs, RAFS v6 offers superior performance through kernel-native paths, whereas RAFS v5 provides broader kernel compatibility.

Can I use Nydus with standard container runtimes like containerd?

Yes. Nydus integrates with standard container runtimes through the snapshotter interface. The nydus-snapshotter component prepares the POSIX-compatible filesystem stack—whether EROFS or FUSE-based—and mounts it as the container's rootfs. Containerd and other OCI-compatible runtimes then interact with this mount point using standard POSIX calls, unaware of the underlying Nydus acceleration layer. This integration is documented in the repository's containerd setup guides.

Is there a performance difference between EROFS and FUSE paths?

Yes, significant performance differences exist between the two paths. The EROFS path leverages kernel-native VFS operations with zero-copy data paths, eliminating user-space context switches for read operations. As implemented in service/src/block_device.rs, this path provides near-native block device performance. The FUSE path, implemented in service/src/fusedev.rs, requires context switches between kernel and the nydusd daemon for every filesystem operation, introducing higher latency and CPU overhead. For production workloads requiring maximum I/O throughput, the EROFS path is recommended.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}