How AppFlowy Handles Data Persistence and File Storage: Local-First CRDT Architecture

AppFlowy persists structured application data as Collab-CRDT objects in per-workspace RocksDB stores while managing binary assets through a dedicated file-storage service that coordinates temporary local caching, upload task queues, and real-time progress streaming to the Flutter UI.

AppFlowy’s open-source codebase implements a local-first architecture that keeps documents, databases, and user settings available offline using conflict-free replicated data types (CRDTs). According to the AppFlowy-IO/AppFlowy source code, the application separates structured data persistence—handled by the CollabPersistenceImpl layer—from binary file storage, which flows through StorageServiceImpl to manage cloud synchronization and temporary local caching.

Structured Data Persistence with Collab-CRDT and RocksDB

AppFlowy represents all structured application data—documents, databases, folders, and user settings—as Collab objects. Each object is persisted by a CollabPersistenceImpl that writes binary updates into a per-workspace RocksDB key-value store, enabling fast offline access and optional cloud synchronization.

The CollabPersistenceImpl Layer

The CollabPersistenceImpl struct in frontend/rust-lib/collab-integrate/src/collab_builder.rs implements the CollabPersistence trait for loading and saving collab updates. It creates a DataSource::Disk that supplies the collab builder with local storage backed by CollabKVDB from the collab_plugins::local_storage::kv crate.

When initializing a workspace component, the document manager, database manager, and folder manager each instantiate this persistence layer:

• Document manager → CollabPersistenceImpl::new(self.user_service.collab_db(uid)?, uid, workspace_id) ([document/manager.rs](https://github.com/AppFlowy-IO/AppFlowy/blob/main/frontend/rust-lib/flowy-document/src/manager.rs#L101-L106))
• Database manager → DatabasePersistenceImpl wrapping CollabPersistenceImpl ([database2/manager.rs](https://github.com/AppFlowy-IO/AppFlowy/blob/main/frontend/rust-lib/flowy-database2/src/manager.rs#L729-L740))

Loading Data from Disk

When opening a document or database, the builder invokes load_collab_from_disk, which reads the latest updates from RocksDB using a read transaction:

let collab_db = self.db.upgrade().ok_or_else(|| CollabError::Internal(anyhow!("collab_db is dropped")))?;
let object_id = collab.object_id().to_string();
let rocksdb_read = collab_db.read_txn();
if rocksdb_read.is_exist(self.uid, &workspace_id, &object_id) {
    let mut txn = collab.transact_mut();
    rocksdb_read.load_doc_with_txn(self.uid, &workspace_id, &object_id, &mut txn)?;
    txn.commit();
}

Source: [collab_builder.rs](https://github.com/AppFlowy-IO/AppFlowy/blob/main/frontend/rust-lib/collab-integrate/src/collab_builder.rs#L24-L65)

Saving Data to Disk

Upon modification, save_collab_to_disk encodes the collab updates and writes them to RocksDB using a write transaction:

let write_txn = collab_db.write_txn();
write_txn.flush_doc(self.uid, &workspace_id, object_id, encoded_collab)?;

Source: [collab_builder.rs](https://github.com/AppFlowy-IO/AppFlowy/blob/main/frontend/rust-lib/collab-integrate/src/collab_builder.rs#L68-L84)

Creating a New Document with Persistence

To create a document with full persistence support, initialize the CollabBuilder with a CollabPersistenceImpl data source:

use collab_integrate::CollabBuilderConfig;
use flowy_document::manager::DocumentManager;

async fn create_document(uid: i64, workspace_id: Uuid) -> anyhow::Result<()> {
    let collab_db = user_service.collab_db(uid)?;
    let persistence = CollabPersistenceImpl::new(
        Arc::downgrade(&collab_db),
        uid,
        workspace_id,
    );

    let builder = CollabBuilder::new(CollabBuilderConfig::default())
        .with_persistence(persistence.into_data_source());

    let mut doc_manager = DocumentManager::new(uid, workspace_id, builder)?;
    doc_manager.create_new_document().await?;
    Ok(())
}

Key components: CollabPersistenceImpl::new → into_data_source → CollabBuilder configuration as defined in collab_builder.rs.

File Storage Service for Binary Assets

Binary assets—images, attachments, and exported files— bypass the CRDT layer and flow through a dedicated file-storage service. The StorageServiceImpl in frontend/rust-lib/flowy-storage/src/manager.rs coordinates temporary local storage, upload task queues, progress notification, and cloud interaction (AF Cloud).

Core Storage Components

Component	Responsibility
StorageServiceImpl	Core implementation handling upload, download, delete, and progress tracking.
FileTempStorage	Holds temporary local copies of files before cloud upload.
UploadTaskQueue	Serializes and retries upload tasks while respecting storage limits.
StorageCloudService	Abstracts the remote cloud API (GET/PUT/DELETE).

Upload Workflow and Progress Streaming

When the UI triggers a file upload via FileStorageEventUploadFile, StorageServiceImpl::create_upload generates a pre-signed URL and a FileProgressReceiver:

pub async fn create_upload(
    &self,
    workspace_id: &str,
    parent_dir: &str,
    file_path: &str,
) -> Result<(CreatedUpload, Option<FileProgressReceiver>), FlowyError> {
    let upload = self.cloud_service.create_upload(...).await?;
    let notifier = self.progress_notifiers
        .entry(upload.file_id.clone())
        .or_insert_with(|| ProgressNotifier::new(upload.file_id.clone()))
        .subscribe();
    Ok((upload, Some(notifier)))
}

Source: [manager.rs](https://github.com/AppFlowy-IO/AppFlowy/blob/main/frontend/rust-lib/flowy-storage/src/manager.rs#L49-L73)

The UploadTaskQueue streams the file from FileTempStorage to the cloud, broadcasting progress updates via ProgressNotifier → FileProgressReceiver.

On the Flutter side, the FileStorageService registers a native stream to receive JSON-encoded FileProgress messages:

final notifier = FileStorageService().onFileProgress(fileUrl: fileUrl);
notifier.addListener(() {
  final progress = notifier.value; // contains progress (0‑1) and error if any
});

Source: file_storage_task.dart L31-L74

Download and Delete Operations

To download a file, download_object fetches bytes from the cloud and writes them to a local path:

let object_value = cloud_service.get_object(url).await?;
let mut file = tokio::fs::OpenOptions::new()
    .create(true).truncate(true).write(true).open(&local_file_path).await?;
file.write(&object_value.raw).await?;

Source: [manager.rs](https://github.com/AppFlowy-IO/AppFlowy/blob/main/frontend/rust-lib/flowy-storage/src/manager.rs#L21-L44)

Deleting a file removes the local temp file, database record, and cloud object:

await FileStorageEventDeleteFile(DeleteFilePB()..url = fileUrl).send();

Handling Storage Limits

The native layer emits StorageNotification.FileStorageLimitExceeded or SingleFileLimitExceeded when quotas are exceeded. The StoreageNotificationListener parses these and forwards FlowyError objects to the UI:

case StorageNotification.FileStorageLimitExceeded:
    onError?.call(FlowyError.fromBuffer(data));

Source: file_storage_listener.dart L35-L42

Cell-Level Persistence in the Dart Layer

For lightweight cell data updates (e.g., text cells), AppFlowy provides a Dart abstraction that forwards changes to the Rust backend. The TextCellDataPersistence class implements CellDataPersistence<String>:

class TextCellDataPersistence implements CellDataPersistence<String> {
  @override
  Future<FlowyError?> save({
    required String viewId,
    required CellContext cellContext,
    required String data,
  }) async {
    final result = CellBackendService.updateCell(
        viewId: viewId, cellContext: cellContext, data: data);
    return result.then((r) => r.fold((_) => null, (err) => err));
  }
}

Source: cell_data_persistence.dart

This delegates to the collab persistence layer, ensuring all data changes ultimately flow through the CRDT-backed RocksDB storage.

Summary

• AppFlowy data persistence and file storage relies on a dual-layer architecture: structured data uses Collab-CRDT objects stored in RocksDB, while binary assets use a separate file-storage service.
• The CollabPersistenceImpl trait implementation writes binary updates to a per-workspace key-value store at frontend/rust-lib/collab-integrate/src/collab_builder.rs, enabling offline-first document and database management.
• File assets are managed by StorageServiceImpl (frontend/rust-lib/flowy-storage/src/manager.rs), which coordinates temporary local caching, upload task queues, and real-time progress streaming to the Flutter UI via FileStorageService.
• All persistence operations are exposed to the Dart frontend through type-safe wrappers, allowing the UI to read, write, and monitor file transfers without managing underlying storage details.

Frequently Asked Questions

How does AppFlowy store documents and databases locally?

AppFlowy stores documents, databases, and folders as Collab CRDT objects serialized into a per-workspace RocksDB key-value store. The CollabPersistenceImpl struct handles the binary encoding and write transactions, ensuring data is available offline immediately after creation.

What storage backend does AppFlowy use for structured data?

The application uses RocksDB via the CollabKVDB abstraction from the collab_plugins::local_storage::kv crate. This provides fast local storage keyed by user ID, workspace ID, and object ID, supporting the CRDT-based collaboration model implemented in collab_builder.rs.

How does AppFlowy handle file uploads and progress tracking?

Binary files flow through StorageServiceImpl, which creates an UploadTask enqueued in an UploadTaskQueue. A ProgressNotifier streams progress updates (0.0 to 1.0) to the Flutter side via FileStorageService.onFileProgress(), enabling real-time UI updates during cloud synchronization.

Can AppFlowy work offline with local data?

Yes. Because structured data is persisted immediately to RocksDB and files are cached in FileTempStorage before upload, users can create, edit, and view content without an internet connection. Changes sync automatically when connectivity is restored through the cloud service abstraction layer.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}