LiteBox Layered Filesystem: How to Stack Writable and Read-Only Storage Backends

The LiteBox layered filesystem (litebox::fs::layered) combines an upper writable layer with a lower read-only or writable layer, automatically migrating files via copy-on-write when modifications occur while presenting a unified POSIX-compatible view.

The LiteBox layered filesystem enables sandboxed environments and efficient storage stacking by merging two independent backends into a single coherent namespace. According to the Microsoft LiteBox source code, this implementation allows developers to overlay a mutable in-memory filesystem on top of a read-only archive, ensuring that original data remains pristine while allowing full read-write operations through automatic copy-on-write migration.

How the LiteBox Layered Filesystem Works

The implementation stacks two independent filesystem backends: an upper layer that is always writable and a lower layer that may be read-only or writable. When a file is accessed, litebox::fs::layered applies a deterministic set of rules to decide which layer handles the operation.

File Access Rules and Shadowing

The filesystem evaluates the existence of a file in both layers to determine visibility and mutability:

• File exists only in the upper layer: The operation is performed directly on the upper layer.
• File exists only in the lower layer: The lower layer is used. If the operation requires writing, the file is migrated (copy-on-write) to the upper layer first.
• File exists in both layers: The upper-layer entry shadows the lower one—only the upper version is visible to the application.
• File is deleted: A tombstone (EntryX::Tombstone) is stored in the root table so the lower file stays on disk but is hidden from the layered view.

Copy-On-Write Migration

When a write operation targets a file that exists only in the lower layer, the filesystem executes the migrate_file_up routine (lines 168–254 in litebox/src/fs/layered.rs). This function reads the lower file, creates the necessary ancestor directories in the upper filesystem, writes the data, and then swaps descriptor entries so future operations hit the upper file while preserving file offsets and node IDs.

Deletion and Tombstones

The filesystem does not physically delete files from the lower layer. Instead, it inserts a tombstone entry in the root directory table (around lines 990–1010 in layered.rs). This ensures that the lower filesystem remains immutable while the layered view correctly reports the file as absent.

Layering Semantics: Read-Only vs. Writable Lower Layers

The LayeringSemantics enum in layered.rs controls how the lower layer behaves during write operations:

pub enum LayeringSemantics {
    /// Lower layer is read‑only – any write triggers copy‑on‑write to the upper layer.
    LowerLayerReadOnly,
    /// Lower layer files are writable – new files are created only in the upper layer,
    /// but existing lower‑layer files can be written to directly.
    LowerLayerWritableFiles,
}

Use LowerLayerReadOnly when the lower layer is an immutable archive (such as a tar file) and you want complete isolation. Use LowerLayerWritableFiles when the lower layer is a writable backing store but you still want new files to appear in the upper layer for organizational purposes.

Core Architecture and Key Components

The implementation delegates all actual I/O to the concrete filesystems you plug in (e.g., an in-memory FS and a read-only tar archive). It maintains a node-info lookup to present a stable inode space that merges the two layers.

FileSystem<Platform, Upper, Lower>

The generic FileSystem struct in layered.rs binds the platform type and the two backend filesystems. It implements the FileSystem trait from litebox/src/fs/mod.rs, providing POSIX-compatible open, read, write, truncate, and rename operations.

RootDir and EntryX

The RootDir structure is a RwLock that maps paths to Arc<EntryX> values. The EntryX enum distinguishes between Upper, Lower, and Tombstone states, enabling the shadowing and deletion logic.

Descriptor Table Integration

The layered filesystem integrates with LiteBox’s global descriptor table. When migrate_file_up executes, it converts an EntryX::Lower into an EntryX::Upper while preserving file descriptors, ensuring that running processes experience a seamless transition without needing to reopen files.

Practical Examples: Using the LiteBox Layered Filesystem

The following snippets demonstrate how to construct and interact with a layered filesystem. These examples use an in-memory filesystem for the upper layer and a read-only tar archive for the lower layer.

Constructing a Layered Filesystem

use litebox::{LiteBox, platform::mock::MockPlatform};
use litebox::fs::{layered, in_mem, tar_ro, FileSystem, Mode, OFlags};

let litebox = LiteBox::new(MockPlatform::new());

// Upper layer: a mutable in‑memory file system.
let upper = in_mem::FileSystem::new(&litebox);

// Lower layer: a read‑only tar archive (packed in the binary).
static TEST_TAR: &[u8] = include_bytes!("test.tar");
let lower = tar_ro::FileSystem::new(&litebox, TEST_TAR.into());

// Build the layered file system with read‑only lower semantics.
let fs = layered::FileSystem::new(
    &litebox,
    upper,
    lower,
    layered::LayeringSemantics::LowerLayerReadOnly,
);

(Source: [layered.rs](https://github.com/microsoft/litebox/blob/main/litebox/src/fs/layered.rs) – constructor new)

Reading a File that Lives Only in the Lower Layer

// `foo` exists in the tar archive (lower layer) only.
let fd = fs.open("foo", OFlags::RDONLY, Mode::RWXU).unwrap();

let mut buf = vec![0u8; 1024];
let n = fs.read(&fd, &mut buf, None).unwrap();
println!("Read {} bytes: {}", n, String::from_utf8_lossy(&buf[..n]));

fs.close(&fd).unwrap();

Writing to Lower-Layer Files (Copy-On-Write)

// Opening for write triggers migration to the upper layer.
let fd = fs.open("foo", OFlags::WRONLY, Mode::RWXU).unwrap();

// After migration, the file lives in `upper`.
fs.write(&fd, b"new data", None).unwrap();
fs.close(&fd).unwrap();

(The migration logic lives in migrate_file_up – see lines 168‑254 of [layered.rs](https://github.com/microsoft/litebox/blob/main/litebox/src/fs/layered.rs).)

Creating New Files in the Upper Layer

let fd = fs.open(
    "/my_new_file",
    OFlags::CREAT | OFlags::WRONLY,
    Mode::RWXU,
).unwrap();

fs.write(&fd, b"hello layered FS", None).unwrap();
fs.close(&fd).unwrap();

Deleting Files with Tombstones

fs.unlink("foo").unwrap(); // `foo` still exists on lower disk but is hidden.
assert!(matches!(
    fs.open("foo", OFlags::RDONLY, Mode::empty()),
    Err(litebox::fs::errors::OpenError::PathError(
        litebox::fs::errors::PathError::NoSuchFileOrDirectory
    ))
));

(Deletion logic and tombstone insertion are around lines 990‑1010 of [layered.rs](https://github.com/microsoft/litebox/blob/main/litebox/src/fs/layered.rs).)

Listing Directories Across Layers

let dir_fd = fs.open("bar", OFlags::RDONLY, Mode::empty()).unwrap();
let entries = fs.read_dir(&dir_fd).unwrap();

for e in entries {
    println!("{} ({:?})", e.name, e.file_type);
}
fs.close(&dir_fd).unwrap();

The read_dir implementation first lists entries from the upper layer, then merges any missing entries from the lower layer, preserving the . and .. entries.

Key Source Files

• litebox/src/fs/layered.rs – Core layered FS implementation containing the FileSystem struct, LayeringSemantics enum, migrate_file_up routine, and open/close logic. (View source)

• litebox/src/fs/mod.rs – Public FileSystem trait definition, OFlags, Mode bitflags, error types, and basic structs used by the layered implementation. (View source)

• litebox/src/fs/tests.rs (section mod layered) – End-to-end tests demonstrating reading, writing, migration, tombstones, and directory handling. (View source)

• litebox/src/fs/in_mem.rs – In-memory filesystem implementation commonly used as the upper writable layer in examples. (View source)

• litebox/src/fs/tar_ro.rs – Read-only tar archive filesystem used as the lower layer in examples. (View source)

Summary

• The LiteBox layered filesystem stacks an upper writable layer atop a lower layer, presenting a unified POSIX-compatible interface.
• Copy-on-write migration automatically moves files from the lower to upper layer when write operations occur, preserving the lower layer's immutability.
• Tombstones hide deleted lower-layer files without modifying the underlying storage, enabling safe deletion semantics in read-only base images.
• Layering semantics (LowerLayerReadOnly vs LowerLayerWritableFiles) control whether writes trigger migration or pass through to the lower layer.
• The implementation delegates all I/O to pluggable backends (e.g., in_mem::FileSystem, tar_ro::FileSystem) and maintains stable inodes via a merged node-info lookup.

Frequently Asked Questions

What is the primary use case for the LiteBox layered filesystem?

The primary use case is sandboxed execution environments where you need to overlay mutable state on top of immutable base images. For example, you can mount a read-only tar archive as the lower layer and an in-memory filesystem as the upper layer, allowing applications to read from the archive while writing modifications to memory without altering the original archive.

How does copy-on-write affect performance when writing to lower-layer files?

When you open a lower-layer file for writing, the migrate_file_up function (lines 168–254 in layered.rs) performs a full read of the lower file and writes it to the upper layer before allowing your write to proceed. This introduces a one-time latency penalty proportional to the file size during the first write operation. Subsequent operations on that file target the upper layer directly and perform at native upper-layer speeds.

Can I use custom filesystem implementations as layers?

Yes. The FileSystem<Platform, Upper, Lower> struct is generic over the upper and lower backend types, requiring only that they implement the FileSystem trait defined in litebox/src/fs/mod.rs. You can plug in any compliant backend, such as the provided in_mem::FileSystem, tar_ro::FileSystem, or your own implementation providing open, read, write, and directory operations.

Is the LiteBox layered filesystem POSIX compatible?

Yes. The implementation provides a POSIX-compatible view where programs can use standard operations (open, read, write, truncate, rename, unlink, readdir) without knowing whether data resides in the upper or lower layer. The filesystem handles descriptor preservation during migration and maintains stable inode numbers across the merged namespace, ensuring compatibility with applications expecting standard Unix filesystem semantics.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}