How to Manage .palmier Project File I/O Off the Main Thread

All filesystem operations for .palmier packages run on background threads using non-isolated FileIO helpers and a main-actor coordinator that queues mutations, ensuring the UI remains responsive while handling large media files.

Palmier Pro uses a bundled folder-style package format with the .palmier extension to store project data and media. Because these packages can contain large video and audio assets, any read or write operation must be performed off the main thread to prevent UI freezes. The codebase in palmier-io/palmier-pro implements a strict separation between stateless I/O utilities and a main-actor coordinator that synchronizes mutations.

Stateless I/O Helpers in FileIO.swift

The Sources/PalmierPro/Utilities/FileIO.swift file provides non-isolated utility functions that execute filesystem operations without inheriting @MainActor isolation. These helpers perform atomic writes, staged copies, and size-limit checks while remaining completely stateless.

Key functions include:

• stageData(_:pathExtension:): Writes data to a temporary staging file using atomic options
• prepareStagedFile(from:nextTo:maxBytes:): Validates size limits and prepares files for installation
• installPreparedFile(from:to:): Performs atomic file replacement into the package

// Create a temporary staging file (off-main)
nonisolated static func stageData(_ data: Data, pathExtension: String) throws -> URL {
    let url = temporaryFileURL(pathExtension: pathExtension)
    try data.write(to: url, options: .atomic)   // atomic write
    return url
}

// Prepare a staged file next to a package, then replace atomically
nonisolated static func prepareStagedFile(
    from stagedURL: URL, nextTo packageURL: URL, maxBytes: Int64? = nil) throws -> URL {
    // … copyReplacingDestination is also non-isolated
}

Because these functions are marked nonisolated, you can invoke them from background Tasks or from @MainActor-isolated code that hops to a background executor via Task.detached.

Coordinating Mutations with ProjectPackageCoordinator

The Sources/PalmierPro/Project/ProjectPackageCoordinator.swift file contains the single source of truth for all mutations affecting live .palmier packages. While this class lives on the main actor to observe overall project state, it queues filesystem work and executes it only after pending saves complete.

@MainActor
final class ProjectPackageCoordinator {
    // …
    func performMutation<T: Sendable>(_ operation: @escaping () throws -> T) async throws -> T {
        // If a save is in progress, the mutation is queued.
        // The closure runs on the caller’s context, which can be a background task.
    }
}

When the UI or an Agent tool needs to modify a package, call performMutation and supply a closure that uses the FileIO helpers. The coordinator guarantees ordering and atomicity while the closure executes on a background thread.

Export Workflow Implementation

The Sources/PalmierPro/Export/PalmierProjectExporter.swift demonstrates the complete pattern for managing .palmier project file I/O off the main thread, while Sources/PalmierPro/Agent/Tools/ToolExecutor+Export.swift shows how Agent tools invoke this from detached tasks.

1. Create a staging directory with a .palmier-export-…partial suffix next to the destination
2. Copy media using a buffered 4 MiB loop that checks Task.checkCancellation() and writes directly to a FileHandle
3. Write metadata using JSONEncoder and FileIO.writeData helpers
4. Atomically replace the existing package using FileManager.replaceItemAt

All steps run within a Task.detached block without blocking the main thread:

Task.detached(priority: .utility) {
    try await coordinator.performMutation {
        try PalmierProjectExporter.export(
            projectFile: projectFile,
            manifest: manifest,
            generationLog: log,
            sourceProjectURL: sourceURL,
            to: destinationURL,
            progress: { pct in /* update UI on main actor */ }
        )
    }
}

Complete Off-Main Thread Pattern

Here is the complete implementation pattern for adding a media file to a project from an Agent tool or UI component:

// 1️⃣ Launch a background task
Task.detached(priority: .utility) {
    // 2️⃣ Ask the coordinator to perform the mutation
    try await editor.projectPackageCoordinator.performMutation {
        // 3️⃣ Stage the incoming data (off-main)
        let stagedURL = try FileIO.stageData(mediaData, pathExtension: "mp4")
        // 4️⃣ Prepare the final location next to the package
        let finalURL = try FileIO.prepareStagedFile(
            from: stagedURL,
            nextTo: editor.projectURL,
            maxBytes: 500 * 1024 * 1024   // 500 MiB limit
        )
        // 5️⃣ Install the prepared file atomically
        try FileIO.installPreparedFile(from: finalURL, to: editor.projectURL.appendingPathComponent("Media/newclip.mp4"))
    }
}

This pattern ensures that heavy filesystem work runs on a background thread pool while performMutation synchronizes with any ongoing saves to prevent race conditions.

Key Design Benefits

Concern	Implementation
Main-thread blocking	All I/O helpers are nonisolated; called from background tasks
Atomicity	Staging files written atomically, then moved/replaced in single step
Concurrent saves	ProjectPackageCoordinator tracks savesInProgress and queues mutations
Cancellation	Long-running copy loops check Task.checkCancellation()
Size limits	FileIO validates maxBytes and throws FileIOError.fileTooLarge

Summary

• Use FileIO.swift helpers for all direct filesystem operations; they are nonisolated and safe to call from any context
• Always route package mutations through ProjectPackageCoordinator.performMutation to ensure serialization with ongoing saves
• Launch heavy I/O operations via Task.detached or background actors to keep the main thread responsive
• Follow the stage-copy-replace pattern for atomic updates that prevent data corruption on crash or cancellation
• Check Task.checkCancellation() in long-running loops to support clean operation aborts

Frequently Asked Questions

What happens if a save is already in progress when I call performMutation?

The ProjectPackageCoordinator tracks active saves using an internal savesInProgress counter. If a save is in progress, the mutation closure is queued and executes only after the current save completes, preventing race conditions on the .palmier package files.

Why are FileIO helpers non-isolated instead of using a background actor?

Marking helpers as nonisolated allows them to run on whatever executor the caller provides without forcing a specific concurrency context. This flexibility enables both Task.detached blocks and custom serial queues to use the same utilities without actor hopping overhead, while keeping the code stateless and testable.

How does atomic replacement prevent data corruption?

The export workflow writes all data to a staging directory first, then uses FileManager.replaceItemAt to swap the old package with the new one in a single filesystem operation. This ensures that the .palmier package is never in a partially written state; either the entire new package exists or the old one remains intact.

Can I call FileIO methods directly without using ProjectPackageCoordinator?

While you can call FileIO methods directly from background tasks for read operations, you should always use ProjectPackageCoordinator.performMutation for write mutations to ensure coordination with other save operations. Higher-level APIs in Sources/PalmierPro/Backend/BackendStorage.swift follow this pattern by routing all writes through the coordinator.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}