How the Nydus Prefetch Mechanism Accelerates Container Cold Starts

The Nydus prefetch mechanism is a two-stage build-time and runtime system that preloads critical container files before they are requested, converting random I/O during cold starts into sequential background reads that hit local cache instead of remote storage.

The dragonflyoss/nydus repository implements a high-performance container image acceleration solution. Its prefetch mechanism addresses the latency penalty of on-demand fetching by preparing data ahead of time, ensuring that when a container actually reads a file, the content is already available in the local blobcache.

How the Nydus Prefetch Mechanism Works

The implementation spans both image construction and container runtime, coordinated through a prefetch table that guides background workers.

Build-Time Prefetch Table Generation

During image construction, the builder/src/core/prefetch.rs module handles policy parsing and table generation. The Prefetch struct accepts a PrefetchPolicy enum with three variants: Fs, Blob, or None.

When the Fs policy is active, the builder reads a list of absolute paths from STDIN or a file and constructs an ordered map of prefetch patterns. It then generates a prefetch table containing inode numbers (for v5 format) or calculated node IDs (for v6 format) that identify exactly which files should be prioritized at runtime. This table is embedded directly into the RAFS bootstrap metadata, allowing nydusd to discover prefetch hints without external configuration files.

For the Blob policy, the builder skips fine-grained file tracking and instead marks entire blob regions for sequential readahead, which is useful when workload patterns are linear and predictable.

Runtime Prefetch Workers

At runtime, nydusd reads the embedded prefetch table or accepts dynamic hints via the --prefetch-files argument. The daemon initializes a pool of background workers controlled by the threads_count parameter in the RAFS configuration.

Each worker receives BlobPrefetchRequest structs defined in storage/src/device.rs. These requests specify a byte range [offset, offset+size) within a specific blob. For Fs prefetch, the daemon performs an inode lookup to determine which chunks back the requested files, merges adjacent chunk reads according to the merging_size parameter, and issues the read asynchronously.

The fetched data is stored in the blobcache (if enabled), ensuring that when the container process actually issues a read() syscall, the data is served from local memory rather than triggering a synchronous remote fetch. For Blob prefetch, the daemon issues large sequential reads without chunk metadata overhead, maximizing throughput for linear access patterns.

Performance Benefits of Prefetching

The Nydus prefetch mechanism delivers measurable improvements through several optimization strategies:

• Reduced cold-start latency: Critical files are resident in blobcache before the container accesses them, converting remote I/O into local memory reads.
• Backend request merging: The merging_size parameter coalesces multiple small chunk reads into larger contiguous requests, minimizing round-trip overhead and improving throughput.
• Parallel fetching: The threads_count configuration allows concurrent background reads, overlapping network latency with other initialization work.
• Bandwidth control: The bandwidth_rate parameter throttles total prefetch traffic, preventing background operations from saturating the network and interfering with foreground container I/O.
• Flexible policy selection: Build-time embedding suits stable workloads, while runtime --prefetch-files adapts to dynamic or exploratory use cases without image rebuilds.

Configuring Prefetch Policies

Build-Time Configuration (Fs Policy)

To embed a prefetch table during image construction, specify the fs policy and pipe the target paths to STDIN:

nydus-image create \
    --output-dir /tmp/nydus \
    --prefetch-policy fs \
    < <(printf "/bin/sh\n/usr/lib/libc.so.6")

The builder parses these paths in builder/src/core/prefetch.rs, resolves them to inode numbers, and serializes the table into the RAFS bootstrap.

Runtime Dynamic Prefetch

For scenarios requiring ad-hoc prefetching without rebuilding the image, pass paths directly to nydusd:

nydusd --fs-type rafs \
    --mount-point /run/containerd/mounts/12345 \
    --prefetch-files /etc/resolv.conf /etc/hosts

The daemon converts these paths to BlobPrefetchRequest objects and queues them for the background worker pool.

Key Source Files

File	Role	Link
docs/prefetch.md	High-level description of prefetch policies and configuration	https://github.com/dragonflyoss/nydus/blob/master/docs/prefetch.md
builder/src/core/prefetch.rs	Parses user patterns, builds prefetch tables for v5/v6, defines PrefetchPolicy	https://github.com/dragonflyoss/nydus/blob/master/builder/src/core/prefetch.rs
storage/src/device.rs	Defines BlobPrefetchRequest, implements async prefetch workers, exposes prefetch() API	https://github.com/dragonflyoss/nydus/blob/master/storage/src/device.rs
utils/src/metrics.rs	Records prefetch request count, data amount, latency, and bandwidth metrics	https://github.com/dragonflyoss/nydus/blob/master/utils/src/metrics.rs
misc/prefetch_seq_diagram.jpg	Visual sequence diagram of the prefetch workflow	https://github.com/dragonflyoss/nydus/blob/master/misc/prefetch_seq_diagram.jpg

Summary

• The Nydus prefetch mechanism operates in two stages: build-time table generation in builder/src/core/prefetch.rs and runtime background fetching via nydusd.
• Fs policy tracks individual files by inode, while Blob policy performs sequential readahead on entire blob regions.
• Runtime workers process BlobPrefetchRequest objects to populate the blobcache before the container accesses the data, eliminating cold-start latency.
• Request merging (merging_size), parallel threads (threads_count), and bandwidth throttling (bandwidth_rate) optimize backend efficiency.
• Metrics in utils/src/metrics.rs expose prefetch volume, latency, and throughput for observability.

Frequently Asked Questions

What is the difference between Fs and Blob prefetch policies?

The Fs policy operates at the file level. During the build, it resolves user-specified paths to inode numbers (v5) or node IDs (v6) and stores them in a prefetch table. At runtime, nydusd looks up the chunk map for these inodes and fetches the specific data blocks backing those files. This is ideal when you know exactly which files are critical for startup.

The Blob policy operates at the storage layer without file metadata. It instructs the daemon to perform large, sequential readahead operations on contiguous regions of a blob. This works best for workloads with predictable linear access patterns or when file-level granularity is unnecessary, as it avoids the overhead of inode lookups and chunk mapping.

How does Nydus merge prefetch requests to improve efficiency?

Nydus coalesces small, adjacent chunk reads into larger backend requests using the merging_size configuration parameter. When the runtime prefetch worker processes a BlobPrefetchRequest, it examines the chunk map for the target file and identifies contiguous byte ranges. Instead of issuing separate network calls for each small chunk, the worker merges them into a single read operation covering the combined range. This reduces round-trip latency, minimizes connection overhead, and maximizes throughput from the backend storage, particularly for small files that share the same blob region.

Can I enable prefetching on an already built Nydus image?

Yes. While embedding a prefetch table at build-time provides static optimization, you can trigger dynamic prefetching at runtime without rebuilding the image. Pass the --prefetch-files argument to nydusd followed by the absolute paths of the files you want to preload. The daemon resolves these paths against the mounted RAFS filesystem, generates BlobPrefetchRequest objects for the underlying chunks, and queues them for the background worker pool. This approach is useful for exploratory workloads, development environments, or when application startup patterns change and you need to optimize cache warmth without regenerating the image.

How do I monitor prefetch performance and bandwidth usage?

Prefetch activity is instrumented in utils/src/metrics.rs, which exposes counters for request volume, total data prefetched, cumulative latency, and bandwidth consumption. You can query these metrics through the daemon's monitoring interface to determine if the threads_count is sufficient to keep up with demand, whether merging_size is effectively reducing request counts, and if bandwidth_rate throttling is necessary to prevent prefetch traffic from overwhelming the backend. Monitoring these values allows you to tune the prefetch configuration for optimal startup latency without sacrificing foreground I/O performance.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}