Virtio-FS and Rate Limiting Configuration in CubeSandbox: A Complete Guide

CubeSandbox configures virtio-fs devices through the --fs command-line flag using comma-separated key-value pairs that define frontend transport settings, backend filesystem behaviors, and optional token-bucket rate limiting for I/O operations and bandwidth.

CubeSandbox, the upstream foundation of cloud-hypervisor, implements a full-featured virtio-fs device that exposes host directories to guests with POSIX semantics. The configuration syntax is defined in hypervisor/vmm/src/config.rs within the FsConfig structure, which parses arguments into frontend transport options, backend BackendFsConfig settings, and an optional RateLimiterConfig. This guide covers every available parameter for tuning performance, security, and resource isolation.

Frontend Configuration: Transport and Device Identity

The frontend options control how the virtio-fs device presents itself to the guest and how it connects to the backend. These parameters are processed during initial argument parsing in FsConfig::SYNTAX (lines 1388-1400).

• tag – The filesystem identifier visible inside the guest (e.g., myshare).
• socket – Path to the vhost-user socket (ignored when native mode is enabled).
• native – Set to on to use the built-in virtio-fs implementation (bypassing vhost-user).
• num_queues – Number of virtqueues for the device (default varies by mode).
• queue_size – Size of each virtqueue (e.g., 1024).
• id – Unique device identifier for management.
• pci_segment – PCI segment assignment for multi-bus topologies.

Native Mode vs. Vhost-User Backend

native=on activates the in-process virtio-fs implementation handled by hypervisor/virtio-devices/src/vhost_user/fs.rs, eliminating the need for a separate virtiofsd daemon. When disabled, the socket parameter must point to a valid vhost-user socket created by an external virtiofsd process.

Backend Configuration: Filesystem Behavior and Security

Backend options govern how the host filesystem is exposed and cached. These populate the BackendFsConfig structure and are validated by VmConfig::validate.

• shared_dir – The host path exported to the guest (e.g., /var/lib/share).
• thread_pool_size – Worker threads for handling filesystem operations.
• cache – Caching strategy: auto, always, never, or none.
• read_only – Force a read-only mount when set to on.
• rlimit_nofile – Per-process open file limit for the daemon.
• xattr and posix_acl – Enable extended attributes and POSIX ACLs.
• xattrmap – Custom xattr mapping file (see virtiofsd/src/passthrough/xattrmap.rs).
• security_label – Enable SELinux security labeling.
• allowed_dirs – Whitelist specific directories accessible to the guest.
• announce_submounts – Advertise sub-mount points to the guest.
• no_readdirplus – Disable READDIRPLUS optimization.
• writeback – Enable write-back caching (requires cache=auto or always).
• allow_direct_io – Permit direct I/O requests from the guest.
• killpriv_v2 – Activate the newer privilege-dropping security check.

Cache Modes and Performance Tuning

The cache parameter determines metadata and data caching behavior. auto enables dynamic caching based on workload, while always aggressively caches for read-heavy workloads. Combine with writeback=on to delay physical writes, improving performance at the cost of durability during host crashes.

Rate Limiting Configuration: Token Bucket Parameters

CubeSandbox implements token-bucket rate limiting through FsConfig::add_ratelimiter_args, which constructs a RateLimiterConfig for the virtio-fs device. This creates two independent throttles: one for I/O operations and one for bandwidth.

Operations Bucket (ops_*)

• ops_size – Maximum operations per refill interval.
• ops_one_time_burst – Initial burst allowance for the first request.
• ops_refill_time – Refill interval in milliseconds.

Bandwidth Bucket (bw_*)

• bw_size – Maximum bytes per refill interval.
• bw_one_time_burst – Initial byte allowance for startup bursts.
• bw_refill_time – Milliseconds between bandwidth token refills.

These parameters prevent a single guest from monopolizing host filesystem resources by limiting throughput to sustainable levels while allowing temporary bursts.

Configuration Validation and Device Creation

The parsing pipeline flows through three critical components:

1. hypervisor/vmm/src/config.rs – FsConfig::parse tokenizes the --fs string and validates syntax against FsConfig::SYNTAX.
2. hypervisor/vmm/src/device_manager.rs (lines 254-2690) – Instantiates the virtio-fs device and binds the parsed configuration to the VMM.
3. hypervisor/virtio-devices/src/vhost_user/fs.rs – Implements the actual device logic using BackendFsConfig.

Validation ensures that native mode and socket paths are mutually exclusive, and that rate-limiting parameters are positive integers when specified.

Practical Configuration Examples

Configure a native virtio-fs device with auto-caching and rate limiting:

let fs_arg = "\
tag=myshare, \
shared_dir=/var/lib/myshare, \
native=on, \
num_queues=4, \
queue_size=1024, \
cache=auto, \
read_only=off, \
rlimit_nofile=4096, \
ops_size=5000, ops_one_time_burst=1000, ops_refill_time=100, \
bw_size=10485760, bw_one_time_burst=5242880, bw_refill_time=200";

let fs_cfg = FsConfig::parse(fs_arg).expect("failed to parse FS config");

Command-line equivalent passed to the CubeSandbox VMM binary:

--fs "tag=myshare,shared_dir=/var/lib/myshare,native=on,num_queues=4,queue_size=1024,\
cache=auto,read_only=off,rlimit_nofile=4096,\
ops_size=5000,ops_one_time_burst=1000,ops_refill_time=100,\
bw_size=10485760,bw_one_time_burst=5242880,bw_refill_time=200"

Summary

• Frontend options (tag, native, num_queues, queue_size) define the transport layer and device identity, selectable between native implementation and vhost-user sockets.
• Backend options (shared_dir, cache, writeback, xattr, security_label) control filesystem semantics, caching strategies, and security policies through BackendFsConfig.
• Rate limiting uses dual token buckets (ops_* and bw_*) to throttle I/O operations and bandwidth independently, configured via FsConfig::add_ratelimiter_args and stored in RateLimiterConfig.
• All parameters are parsed in hypervisor/vmm/src/config.rs and validated before device creation in the VMM's device manager.

Frequently Asked Questions

What is the difference between native and vhost-user virtio-fs in CubeSandbox?

Native mode (native=on) runs the virtio-fs implementation inside the VMM process using internal crates, eliminating socket overhead and simplifying deployment. Vhost-user mode requires an external virtiofsd daemon listening on a Unix socket specified by the socket parameter, isolating filesystem operations in a separate process for security.

How does the token bucket rate limiting work for virtio-fs?

CubeSandbox creates two token buckets: one counting I/O operations and one measuring bandwidth in bytes. Each bucket fills at the rate defined by refill_time, holds a maximum of size tokens, and grants a one_time_burst on initial use. When a request consumes tokens, it waits if the bucket is empty, effectively throttling the guest to the configured limits.

Which cache mode should I use for virtio-fs?

Use cache=auto for general-purpose workloads, as it dynamically balances consistency and performance. Choose cache=always with writeback=on for read-heavy or write-buffered scenarios where durability is less critical than speed. Use cache=none or cache=never for strict consistency requirements where the guest must see immediate host filesystem changes.

Where is the virtio-fs configuration parsed in the CubeSandbox source code?

The configuration string is parsed in hypervisor/vmm/src/config.rs within the FsConfig implementation, specifically around line 1388 where FsConfig::SYNTAX defines the valid grammar. Rate limiting arguments are processed by FsConfig::add_ratelimiter_args, and the final configuration is validated by VmConfig::validate before device instantiation in hypervisor/vmm/src/device_manager.rs.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}