Supported Dataset Formats for Stable-WorldModel: HDF5, Lance, LeRobot, and Folder Layout

Stable-WorldModel supports four interchangeable dataset formats—HDF5 binaries, LanceDB tables, LeRobot Hub datasets, and plain folder structures—each automatically detected via the format registry in stable_worldmodel/data/formats/.

Stable-WorldModel from the galilai-group provides a unified data abstraction layer for robotic world models. The library supports four distinct storage backends that share a common Dataset API, allowing you to read training data from local directories, columnar databases, or remote hubs without changing downstream code. Understanding these supported dataset formats for Stable-WorldModel is essential for building efficient data pipelines.

HDF5 Binary Format (Single-File Containers)

The HDF5 format stores entire datasets in a single binary file with .h5 or .hdf5 extensions. In stable_worldmodel/data/formats/hdf5.py, the HDF5Dataset class implements lazy loading for large arrays, while HDF5Writer handles write modes including 'append', 'overwrite', and 'error'. The HDF5.detect method identifies this format by checking for the file suffix or locating an HDF5 file within a directory.

This format supports remote URIs such as s3://bucket/dataset.h5 via fsspec integration in HDF5Dataset._open_h5, making it suitable for cloud-based training workflows.

from stable_worldmodel.world.world import World

# Auto-detect and open an HDF5 file from S3

dataset = World.dataset_from_path(
    path="s3://my-bucket/dataset.h5",
    storage_options={"anon": False},  # fsspec authentication options

)
print(dataset.get_col_data("action")[:5])

LanceDB Columnar Tables

LanceDB stores data in a column-oriented layout within directories ending in .lance. The implementation in stable_worldmodel/data/formats/lance.py provides LanceDataset for reading and LanceWriter for appending or creating tables. The Lance.detect method recognizes paths ending with .lance or directories containing a .lance subdirectory.

LanceDB format excels at efficient random access and is particularly suitable for datasets requiring fast columnar queries during training.

from stable_worldmodel.data.formats.lance import LanceWriter

# Append episodes to an existing LanceDB table

with LanceWriter("datasets/my_table.lance", mode="append") as writer:
    writer.write_episodes([ep_data1, ep_data2])

LeRobot Hub Datasets (Read-Only)

The LeRobot format provides read-only access to datasets hosted on the LeRobot Hub via the lerobot:// URI scheme. Implemented in stable_worldmodel/data/formats/lerobot.py, the LeRobotAdapter class maps these URIs to underlying Hugging Face datasets. The LeRobot.detect method simply checks for the lerobot:// protocol prefix.

This format is ideal for quickly prototyping world models on public robotic datasets without downloading files locally.

from stable_worldmodel.world.world import World

# Load a dataset directly from the LeRobot Hub

dataset = World.dataset_from_path(
    path="lerobot://my-org/robot-push-v0",
    primary_camera_key="camera_front",
    keys_to_load=["pixels", "action", "proprio"],
)
print(dataset.column_names)  # Output: ['pixels', 'action', 'proprio', 'ep_idx', 'step_idx']

Plain Folder Structure Layout

The Folder format uses a plain filesystem layout where image columns reside in subdirectories and tabular data is stored as compressed NumPy archives (.npz). In stable_worldmodel/data/formats/folder.py, FolderDataset detects this format by looking for ep_len.npz and rejecting video sub-folders, while FolderWriter creates the directory structure including ep_len.npz, ep_offset.npz, and individual image files.

This format stores data as follows:

• ep_len.npz and ep_offset.npz for episode metadata
• .npz files for array columns (e.g., action.npz, state.npz)
• Subdirectories (e.g., pixels/) containing JPEG images named ep_0_step_0.jpeg, etc.

from stable_worldmodel.data.formats.folder import FolderWriter
import numpy as np

ep_data = {
    "action": np.random.randint(0, 5, size=(100,)),
    "state": np.random.randn(100, 10).astype(np.float32),
    "pixels": np.random.randint(0, 255, size=(100, 3, 64, 64), dtype=np.uint8),
}

# Write episode to folder structure

with FolderWriter("my_dataset_root", mode="overwrite") as writer:
    writer.write_episode(ep_data)

The resulting directory structure:

my_dataset_root/
│   ep_len.npz
│   ep_offset.npz
│   action.npz
│   state.npz
└── pixels/
        ep_0_step_0.jpeg
        ep_0_step_1.jpeg
        ...

How Automatic Format Detection Works

Stable-WorldModel uses a registry-based architecture to automatically instantiate the correct reader or writer. Each format class inherits from stable_worldmodel/data/format.py:Format and registers itself using the @register_format decorator. When you call World.dataset_from_path(), the core World class consults this registry and invokes each format's detect method to identify the appropriate backend.

All format-specific readers (FolderDataset, LanceDataset, HDF5Dataset, LeRobotAdapter) subclass stable_worldmodel/data/dataset.py:Dataset, exposing uniform methods like _load_slice, get_col_data, get_row_data, and column_names. This design ensures that switching between an HDF5 file and a folder structure requires no changes to your training loop.

Summary

• Four supported formats: HDF5 (.h5), LanceDB (.lance), LeRobot Hub (lerobot://), and plain folder structures.
• Auto-detection: The @register_format registry in stable_worldmodel/data/formats/ enables automatic backend selection via World.dataset_from_path().
• Unified API: All formats implement the Dataset base class with consistent methods for slicing and column access.
• Write capabilities: HDF5, LanceDB, and Folder formats support writing; LeRobot Hub is read-only.
• Cloud compatible: HDF5 and LanceDB formats support S3 URIs via fsspec integration.

Frequently Asked Questions

What dataset formats can I write to using Stable-WorldModel?

You can write data to HDF5, LanceDB, and Folder formats using their respective writers (HDF5Writer, LanceWriter, and FolderWriter). Each writer supports 'append', 'overwrite', and 'error' modes validated by the validate_write_mode helper. The LeRobot format is read-only and does not support writing.

How does Stable-WorldModel detect which format to use?

The library uses the @register_format decorator to build a registry of available formats. When you provide a path to World.dataset_from_path(), each registered format's detect method is called to check for signatures like .h5 suffixes, .lance directories, lerobot:// prefixes, or the presence of ep_len.npz files for folder layouts.

Can I use cloud storage URLs like S3 with Stable-WorldModel?

Yes. The HDF5 format supports S3 URIs (e.g., s3://bucket/dataset.h5) through fsspec integration in HDF5Dataset._open_h5. You can pass authentication options via the storage_options parameter. The LanceDB format also accepts cloud URIs for remote tables.

Is the LeRobot Hub format suitable for local training?

The LeRobot format is designed for read-only streaming from the Hugging Face Hub via lerobot:// URIs. While convenient for quick prototyping and public datasets, it requires an internet connection and does not cache locally by default, making it less suitable for high-throughput local training compared to HDF5 or LanceDB formats.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}