# How the nydusd FUSE Driver Works: Architecture and Implementation

> Learn how the nydusd FUSE driver operates. Discover its Rust implementation, worker threads, and request loop for high-performance filesystem access.

- Repository: [dragonflyoss/nydus](https://github.com/dragonflyoss/nydus)
- Tags: architecture
- Published: 2026-02-28

---

**The nydusd FUSE driver implements a high-performance, read-only filesystem in Rust by spawning multiple worker threads that pull requests from `/dev/fuse` via a `FuseSession`, dispatching them to a VFS backend through the `FuseServer` request loop.**

The nydusd FUSE driver serves as the primary filesystem interface for the Dragonfly Nydus project, transforming standard Linux mount points into content-addressable container image stores. Written in Rust and built atop the `fuse_backend_rs` library, this driver enables on-demand image data fetching through a multi-threaded architecture optimized for container startup performance.

## Core Architecture Components

The nydusd FUSE driver consists of four tightly-coupled Rust components that together manage the kernel interface, request dispatch, and lifecycle operations.

### FusedevFsService (Session and Backend Management)

The `FusedevFsService` struct, defined in [`service/src/fusedev.rs`](https://github.com/dragonflyoss/nydus/blob/main/service/src/fusedev.rs), serves as the central coordinator for the FUSE filesystem. It holds the **VFS** instance, the `FuseSession`, and the backend collection. This component creates the fuse session, mounts it on the target directory, and provides critical helpers for cache-invalidation and fail-over notifications through methods like `drain_fuse_requests`.

### FuseServer (Request Dispatch Layer)

`FuseServer` acts as a thin wrapper around `fuse_backend_rs::api::Server`. According to the dragonflyoss/nydus source code, its `svc_loop` method (located at [`service/src/fusedev.rs`](https://github.com/dragonflyoss/nydus/blob/main/service/src/fusedev.rs) lines 21-44) continuously pulls raw requests from `/dev/fuse` and forwards them to the VFS for processing. The server owns an `Arc<Server<Vfs>>` and a `FuseChannel` bound to the session.

### FusedevDaemon (Lifecycle and Threading)

The `FusedevDaemon` object represents the daemon instance that the Nydus runtime interacts with. Its `start` method spawns a configurable number of worker threads (controlled by the `threads_cnt` parameter), with each thread running its own `FuseServer` instance. The daemon manages the complete lifecycle including mount, graceful shutdown via `stop`, and cleanup through `wait`.

### create_fuse_daemon Factory Function

The `create_fuse_daemon` convenience factory, implemented in [`service/src/fusedev.rs`](https://github.com/dragonflyoss/nydus/blob/main/service/src/fusedev.rs) (lines 106-122), constructs a fully initialized `FusedevDaemon`. This function mounts the session, stores the connection ID for fail-over tracking, and optionally prepares live-upgrade handles when API sockets are configured.

## Request Processing Pipeline

The nydusd FUSE driver processes filesystem operations through a six-stage pipeline that bridges kernel requests to the RAFS backend.

1. **Session Creation**: When `FusedevFsService::new` instantiates the service, it creates a `FuseSession` on the target mount point using "rafs" as the registered filesystem name. The session abstracts the kernel-side `/dev/fuse` device and stores the connection ID used for fail-over operations.

2. **Server Initialization**: `FusedevFsService::create_fuse_server` builds a `FuseServer` that owns an `Arc<Server<Vfs>>` and binds it to a `FuseChannel` connected to the session.

3. **Worker Thread Spawning**: `FusedevDaemon::start` spawns *N* threads based on the `threads_cnt` configuration. Each thread executes `kick_one_server`, which creates a fresh `FuseServer`, registers a `FuseOpWrapper` metrics hook to record inflight operations, and enters the service loop.

4. **Request Loop Execution**: Inside `FuseServer::svc_loop`, each worker thread repeatedly calls `self.ch.get_request()` to retrieve the next request from the kernel. The thread hands the request to `self.server.handle_message`, passing the optional metrics hook. The loop handles `EncodeMessage` errors (indicating kernel session closure) and continues until receiving `EBADF` or an explicit `umount` call.

5. **Cache Invalidation and Fail-over**: When the daemon flushes stale data after a live-upgrade, `FusedevFsService::drain_fuse_requests` executes. Depending on the configured `FailoverPolicy`, it either writes to `/proc/sys/fs/fuse/connections/*/resend` via `FuseSysfsNotifier` or sends resend notifications directly through the `/dev/fuse` file descriptor using `FusedevNotifier`.

6. **Lifecycle Termination**: `FusedevFsService::umount` gracefully shuts down the `FuseSession` and wakes the channel. `FusedevDaemon::stop` forces the session's `wake()` method, causing all worker threads to exit their loops, while `FusedevDaemon::wait` joins all threads to ensure clean shutdown.

## Starting a nydusd FUSE Daemon

The following example demonstrates how to instantiate the nydusd FUSE driver using the factory function:

```rust
use std::sync::Arc;
use fuse_backend_rs::api::Vfs;
use nydus_api::BuildTimeInfo;
use nydus_rafs::metadata::RafsSuperBlock;
use nydus_runtime::service::create_fuse_daemon;
use nydus_runtime::upgrade::FailoverPolicy;
use mio::Waker;

fn main() -> anyhow::Result<()> {
    // 1. Build a VFS (RAFS backend in this example)
    let vfs = create_vfs_backend(
        FsBackendType::Rafs,
        /* is_fuse = */ true,
        /* hybrid_mode = */ false,
    )?;

    // 2. Prepare runtime helpers
    let waker = Arc::new(Waker::new(poll::Poll::new()?.registry(), 0)?);
    let bti = BuildTimeInfo::default();

    // 3. Create and start the fuse daemon
    let daemon = create_fuse_daemon(
        "/tmp/nydus_mount",   // mount point
        vfs,
        None,                 // no supervisor socket
        Some("demo".into()), // daemon ID
        4,                    // number of worker threads
        waker,
        None,                 // no API socket (no live‑upgrade)
        false,                // upgrade disabled
        false,                // read‑write mode
        FailoverPolicy::None,
        None,                 // no explicit backend mount command
        bti,
    )?;

    // 4. The daemon is now serving the RAFS image via FUSE.
    //    It runs in background threads; block the main thread if needed.
    daemon.wait()?;
    Ok(())
}

```

This snippet mirrors the real factory call used throughout the dragonflyoss/nydus project, configuring four worker threads and read-only mode.

## Key Source Files

| File | Description |
|------|-------------|
| **service/src/fusedev.rs** | Core FUSE driver implementation containing `FusedevFsService`, `FuseServer`, `FusedevDaemon`, and `create_fuse_daemon`. |
| **service/src/fs_service.rs** | Definition of the `FsService` trait that `FusedevFsService` implements for mount, upgrade, and cache-invalidation operations. |
| **service/src/daemon.rs** | Generic daemon infrastructure including the state machine and channel plumbing that `FusedevDaemon` extends. |
| **service/src/vfs.rs** | VFS wrapper around `fuse_backend_rs::api::Vfs` that handles the actual request processing logic. |
| **rafs/** | RAFS image parsing and inode structures (e.g., `metadata::RafsInode`) used by the VFS to resolve filesystem operations. |

## Summary

- The nydusd FUSE driver architecture centers on `FusedevFsService` for session management, `FuseServer` for request handling, and `FusedevDaemon` for thread lifecycle control.
- Worker threads execute `FuseServer::svc_loop` to pull requests from `/dev/fuse` and dispatch them to the VFS backend.
- The driver registers itself with the kernel using the "rafs" filesystem type and maintains connection IDs for fail-over scenarios.
- Cache invalidation supports both sysfs-based (`FuseSysfsNotifier`) and direct descriptor (`FusedevNotifier`) notification methods depending on the `FailoverPolicy`.
- Graceful shutdown is coordinated through `FusedevDaemon::stop` and `wait`, ensuring all worker threads exit cleanly before the process terminates.

## Frequently Asked Questions

### What is the role of FusedevFsService in the nydusd FUSE driver?

`FusedevFsService` serves as the primary orchestrator located in [`service/src/fusedev.rs`](https://github.com/dragonflyoss/nydus/blob/main/service/src/fusedev.rs). It holds the VFS instance, manages the `FuseSession` connected to `/dev/fuse`, and provides the `drain_fuse_requests` method for cache invalidation during live upgrades.

### How does nydusd handle concurrent FUSE requests?

The driver handles concurrency through `FusedevDaemon`, which spawns a configurable number of worker threads (set via `threads_cnt`) during startup. Each thread runs an independent `FuseServer::svc_loop` that pulls requests from the kernel, allowing parallel processing of filesystem operations against the RAFS backend.

### What filesystem name does nydusd register with the kernel?

According to the source code in [`service/src/fusedev.rs`](https://github.com/dragonflyoss/nydus/blob/main/service/src/fusedev.rs), nydusd registers the filesystem name **"rafs"** when creating the `FuseSession`. This identifier appears in mount listings and kernel logs when the driver is active.

### How does the driver handle fail-over during live upgrades?

During live upgrades, `FusedevFsService::drain_fuse_requests` executes based on the configured `FailoverPolicy`. The driver either writes to `/proc/sys/fs/fuse/connections/*/resend` using `FuseSysfsNotifier` or sends direct notifications through the `/dev/fuse` file descriptor using `FusedevNotifier` to force the kernel to resend pending requests to the new daemon instance.