How to Implement AI Features in a macOS Video Editor: A Complete Guide to Palmier Pro

Implement AI features in a macOS video editor by routing user actions through a three-layer architecture: SwiftUI menus trigger ViewModel orchestration, which dispatches asynchronous jobs to an AI agent via MCP, returning placeholder assets that are replaced upon completion.

Palmier Pro demonstrates how to implement AI features in a macOS video editor using a modular, asynchronous pipeline that supports upscaling, audio generation, and video creation directly from the timeline. The architecture splits responsibilities between the UI layer, ViewModel orchestration, and a remote AI agent, ensuring the editor remains responsive during long-running operations. This technical deep dive reveals the exact source files and patterns used to bridge native SwiftUI interfaces with remote AI services.

Understanding the Three-Layer AI Architecture

UI and Interaction Layer

The UI layer handles user entry points through SwiftUI context menus and inspector panels. In Sources/PalmierPro/Generation/Edit/AIEditMenu.swift, context menus expose AI actions like upscaling or editing based on the selected media type. The timeline integration in Sources/PalmierPro/Timeline/TimelineView+AIEditMenu.swift provides AppKit-based fallback menus for right-click interactions on clips. For detailed parameter controls, Sources/PalmierPro/Inspector/Tabs/AIEditTab.swift renders inspector panels with model selection and prompt inputs.

View-Model Orchestration Layer

The ViewModel layer manages state and determines permitted actions. Sources/PalmierPro/Editor/ViewModel/EditorViewModel+AIEdit.swift tracks pending operations through showGenerationPanel and pendingPanelSeed states. When a user selects an action, the ViewModel validates the current selection and delegates to EditSubmitter for request construction. The clipReplacementHandlers (lines 13-28 in EditorViewModel+AIEdit.swift) manage the transition from placeholder to final asset, calling replaceClipMediaRef(clipId:newAssetId:resetTrim:) once generation completes.

Backend and Agent Layer

The backend layer communicates with remote AI services via the Model Context Protocol (MCP). Sources/PalmierPro/Agent/Clients/PalmierClient.swift sends GenerationInput objects to the Palmier Pro AI agent. Sources/PalmierPro/Agent/Tools/ToolDefinitions.swift declares available capabilities like upscale_media or generate_video with cost metadata and input schemas. The agent returns immediate placeholder IDs while Sources/PalmierPro/Editor/ProjectActivityView.swift displays progress, and Sources/PalmierPro/Editor/ViewModel/EditorViewModel+Cost.swift persists generation logs to generation-log.json.

Implementing the AI Feature Flow

The end-to-end flow follows these discrete steps:

1. User invokes an AI action from the timeline right-click menu (TimelineView+AIEditMenu.swift) or media panel context menu (AIEditMenu.swift).

2. Menu forwards to ViewModel by calling methods like runUpscale(_:) or EditSubmitter.submitUpscale(asset:model:editor:).

3. Submitter builds GenerationInput in Sources/PalmierPro/Generation/Edit/EditSubmitter.swift, packing the model ID, source URL, trim information, and optional prompts.

4. Client dispatches to agent via PalmierClient.shared.send(input, editor: editor), which communicates through MCPService to remote providers like Gemini or ElevenLabs.

5. ViewModel updates state by setting pendingEditReplacementClipId and showGenerationPanel = true, displaying a spinner in ProjectActivityView.swift.

6. Agent returns placeholder asset ID immediately, then processes asynchronously. When complete, it writes the new media file to the project bundle and updates generation-log.json.

7. Clip replacement occurs via clipReplacementHandlers, which call replaceClipMediaRef and optionally resetTrim to display the new source fully.

Key Source Files and Responsibilities

• Sources/PalmierPro/Generation/Edit/AIEditMenu.swift – SwiftUI context menus exposing AI actions (upscale, edit, generate audio) with availability checks.

• Sources/PalmierPro/Timeline/TimelineView+AIEditMenu.swift – AppKit-based timeline integration for macOS-native right-click menus on clips.

• Sources/PalmierPro/Editor/ViewModel/EditorViewModel+AIEdit.swift – Orchestrates AI actions, manages pendingEditReplacementClipId, and handles clip replacement logic.

• Sources/PalmierPro/Generation/Edit/EditSubmitter.swift – Factory for GenerationInput objects; validates inputs and dispatches to PalmierClient.

• Sources/PalmierPro/Agent/Tools/ToolDefinitions.swift – Declares remote tool specifications including name, description, inputs, and cost metadata.

• Sources/PalmierPro/Agent/Clients/PalmierClient.swift – Low-level MCP client handling request serialization and response parsing.

• Sources/PalmierPro/Editor/ViewModel/EditorViewModel+Cost.swift – Persists generation logs and enforces subscription limits through AccountService.shared.isSignedIn.

Adding a Custom AI Capability: Step-by-Step Example

To add AI Stabilization for shaky footage, extend the existing pipeline:

Step 1: Define the remote tool in ToolDefinitions.swift:

// Inside Sources/PalmierPro/Agent/Tools/ToolDefinitions.swift
static let stabilize = ToolDefinition(
    name: "stabilize_video",
    description: "Stabilizes shaky video footage using AI. Returns a placeholder asset ID.",
    inputs: [.media, .optionalPrompt],
    cost: .perSecond(0.03)
)

Step 2: Create the UI entry in AIEditMenu.swift:

if availableActions.contains(.stabilize) {
    Button("Stabilize") { stabilize() }
}

Step 3: Update EditAction availability to include .stabilize for video assets in the enum definition.

Step 4: Implement the submitter in EditSubmitter.swift:

static func submitStabilize(asset: MediaAsset,
                            editor: EditorViewModel) -> Bool {
    let input = GenerationInput(
        model: ToolDefinitions.stabilize.name,
        mediaRef: asset.id,
        prompt: nil
    )
    return PalmierClient.shared.send(input, editor: editor)
}

Step 5: Handle completion using the existing clipReplacementHandlers infrastructure, which automatically replaces the clip once the placeholder resolves.

Handling Asynchronous Completion and Clip Replacement

The EditorViewModel uses a closure-based completion system to handle asynchronous AI results. In EditorViewModel+AIEdit.swift, the clipReplacementHandlers method returns paired completion and failure closures:

private func clipReplacementHandlers(clipId: String,
                                    resetTrim: Bool)
    -> (onComplete: (@MainActor (MediaAsset) -> Void)?,
        onFailure: (@MainActor () -> Void)?) {
    markPendingReplacement(clipId: clipId)
    let fired = FirstOnlyFlag()
    let onComplete: @MainActor (MediaAsset) -> Void = { [weak self] newAsset in
        guard fired.fire() else { return }
        self?.replaceClipMediaRef(clipId: clipId,
                                  newAssetId: newAsset.id,
                                  resetTrim: resetTrim)
        self?.clearPendingReplacement(clipId: clipId)
    }
    return (onComplete, nil)
}

The FirstOnlyFlag ensures the replacement logic executes exactly once, preventing race conditions if the agent sends multiple status updates. The resetTrim parameter clears any in-out points when replacing source media, ensuring the new stabilized footage displays fully.

Summary

• AI actions are declarative – The UI layer only decides what to do; EditSubmitter creates a GenerationInput, and the remote agent handles execution.

• State is centralized in EditorViewModel; pending operations, trim handling, and cost logging live in dedicated extensions, making new features additive rather than invasive.

• Extending the pipeline requires three steps: (1) a new ToolDefinition in ToolDefinitions.swift, (2) a UI entry in AIEditMenu.swift or AIEditTab.swift, and (3) a thin wrapper in EditSubmitter.

• Asynchronous by design – Placeholder assets keep the UI responsive while clipReplacementHandlers manage the final swap when remote jobs complete.

• Security and billing – Every tool includes cost metadata; EditorViewModel+Cost.swift persists logs for usage tracking and subscription enforcement.

Frequently Asked Questions

How does Palmier Pro handle AI generation latency without blocking the UI?

Palmier Pro implements an asynchronous placeholder pattern. When a user triggers an AI action, PalmierClient immediately returns a placeholder asset ID while dispatching the actual work to the remote AI agent. The ViewModel sets showGenerationPanel = true and updates ProjectActivityView.swift with a spinner. The editor remains fully interactive; when the agent completes generation, clipReplacementHandlers automatically swaps the placeholder for the final media file.

What is the role of the MCP server in Palmier Pro's AI architecture?

The Model Context Protocol (MCP) server acts as the bridge between the native macOS application and remote AI providers like Gemini or ElevenLabs. PalmierClient.swift serializes GenerationInput objects into MCP requests, while ToolDefinitions.swift declares the available tool schemas. This abstraction allows the video editor to support multiple AI providers through a unified interface without changing the UI or ViewModel layers.

How do I add a new AI model to the existing pipeline?

Adding a new model requires updating ToolDefinitions.swift with a new ToolDefinition specifying the model name, required inputs, and cost structure. Then add a menu item in AIEditMenu.swift and a corresponding submission method in EditSubmitter.swift that constructs the appropriate GenerationInput. The existing clipReplacementHandlers in EditorViewModel+AIEdit.swift handle completion automatically, so no additional completion logic is necessary unless the feature requires custom UI feedback.

Where does Palmier Pro store generation history and cost tracking?

Generation history persists in generation-log.json within the project bundle, managed by Sources/PalmierPro/Editor/ViewModel/EditorViewModel+Cost.swift. This file records each AI operation including tool usage, token costs, and timestamps. The ViewModel exposes this data to the UI for project activity displays and integrates with AccountService.shared.isSignedIn to enforce subscription limits and usage quotas.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}