# How to Implement AI Features in a macOS Video Editor: A Complete Guide to Palmier Pro

> Learn to implement AI features in a macOS video editor using a three-layer architecture. Discover how SwiftUI menus, ViewModel, and an AI agent streamline your workflow for placeholder asset replacement.

- Repository: [Palmier/palmier-pro](https://github.com/palmier-io/palmier-pro)
- Tags: how-to-guide
- Published: 2026-06-24

---

**Implement AI features in a macOS video editor by routing user actions through a three-layer architecture: SwiftUI menus trigger ViewModel orchestration, which dispatches asynchronous jobs to an AI agent via MCP, returning placeholder assets that are replaced upon completion.**

Palmier Pro demonstrates how to implement AI features in a macOS video editor using a modular, asynchronous pipeline that supports upscaling, audio generation, and video creation directly from the timeline. The architecture splits responsibilities between the UI layer, ViewModel orchestration, and a remote AI agent, ensuring the editor remains responsive during long-running operations. This technical deep dive reveals the exact source files and patterns used to bridge native SwiftUI interfaces with remote AI services.

## Understanding the Three-Layer AI Architecture

### UI and Interaction Layer

The **UI layer** handles user entry points through SwiftUI context menus and inspector panels. In [`Sources/PalmierPro/Generation/Edit/AIEditMenu.swift`](https://github.com/palmier-io/palmier-pro/blob/main/Sources/PalmierPro/Generation/Edit/AIEditMenu.swift), context menus expose AI actions like upscaling or editing based on the selected media type. The timeline integration in `Sources/PalmierPro/Timeline/TimelineView+AIEditMenu.swift` provides AppKit-based fallback menus for right-click interactions on clips. For detailed parameter controls, [`Sources/PalmierPro/Inspector/Tabs/AIEditTab.swift`](https://github.com/palmier-io/palmier-pro/blob/main/Sources/PalmierPro/Inspector/Tabs/AIEditTab.swift) renders inspector panels with model selection and prompt inputs.

### View-Model Orchestration Layer

The **ViewModel layer** manages state and determines permitted actions. `Sources/PalmierPro/Editor/ViewModel/EditorViewModel+AIEdit.swift` tracks pending operations through `showGenerationPanel` and `pendingPanelSeed` states. When a user selects an action, the ViewModel validates the current selection and delegates to `EditSubmitter` for request construction. The `clipReplacementHandlers` (lines 13-28 in `EditorViewModel+AIEdit.swift`) manage the transition from placeholder to final asset, calling `replaceClipMediaRef(clipId:newAssetId:resetTrim:)` once generation completes.

### Backend and Agent Layer

The **backend layer** communicates with remote AI services via the Model Context Protocol (MCP). [`Sources/PalmierPro/Agent/Clients/PalmierClient.swift`](https://github.com/palmier-io/palmier-pro/blob/main/Sources/PalmierPro/Agent/Clients/PalmierClient.swift) sends `GenerationInput` objects to the Palmier Pro AI agent. [`Sources/PalmierPro/Agent/Tools/ToolDefinitions.swift`](https://github.com/palmier-io/palmier-pro/blob/main/Sources/PalmierPro/Agent/Tools/ToolDefinitions.swift) declares available capabilities like `upscale_media` or `generate_video` with cost metadata and input schemas. The agent returns immediate placeholder IDs while [`Sources/PalmierPro/Editor/ProjectActivityView.swift`](https://github.com/palmier-io/palmier-pro/blob/main/Sources/PalmierPro/Editor/ProjectActivityView.swift) displays progress, and `Sources/PalmierPro/Editor/ViewModel/EditorViewModel+Cost.swift` persists generation logs to [`generation-log.json`](https://github.com/palmier-io/palmier-pro/blob/main/generation-log.json).

## Implementing the AI Feature Flow

The end-to-end flow follows these discrete steps:

1. **User invokes an AI action** from the timeline right-click menu (`TimelineView+AIEditMenu.swift`) or media panel context menu ([`AIEditMenu.swift`](https://github.com/palmier-io/palmier-pro/blob/main/AIEditMenu.swift)).

2. **Menu forwards to ViewModel** by calling methods like `runUpscale(_:)` or `EditSubmitter.submitUpscale(asset:model:editor:)`.

3. **Submitter builds GenerationInput** in [`Sources/PalmierPro/Generation/Edit/EditSubmitter.swift`](https://github.com/palmier-io/palmier-pro/blob/main/Sources/PalmierPro/Generation/Edit/EditSubmitter.swift), packing the model ID, source URL, trim information, and optional prompts.

4. **Client dispatches to agent** via `PalmierClient.shared.send(input, editor: editor)`, which communicates through `MCPService` to remote providers like Gemini or ElevenLabs.

5. **ViewModel updates state** by setting `pendingEditReplacementClipId` and `showGenerationPanel = true`, displaying a spinner in [`ProjectActivityView.swift`](https://github.com/palmier-io/palmier-pro/blob/main/ProjectActivityView.swift).

6. **Agent returns placeholder asset ID** immediately, then processes asynchronously. When complete, it writes the new media file to the project bundle and updates [`generation-log.json`](https://github.com/palmier-io/palmier-pro/blob/main/generation-log.json).

7. **Clip replacement occurs** via `clipReplacementHandlers`, which call `replaceClipMediaRef` and optionally `resetTrim` to display the new source fully.

## Key Source Files and Responsibilities

- **[`Sources/PalmierPro/Generation/Edit/AIEditMenu.swift`](https://github.com/palmier-io/palmier-pro/blob/main/Sources/PalmierPro/Generation/Edit/AIEditMenu.swift)** – SwiftUI context menus exposing AI actions (upscale, edit, generate audio) with availability checks.

- **`Sources/PalmierPro/Timeline/TimelineView+AIEditMenu.swift`** – AppKit-based timeline integration for macOS-native right-click menus on clips.

- **`Sources/PalmierPro/Editor/ViewModel/EditorViewModel+AIEdit.swift`** – Orchestrates AI actions, manages `pendingEditReplacementClipId`, and handles clip replacement logic.

- **[`Sources/PalmierPro/Generation/Edit/EditSubmitter.swift`](https://github.com/palmier-io/palmier-pro/blob/main/Sources/PalmierPro/Generation/Edit/EditSubmitter.swift)** – Factory for `GenerationInput` objects; validates inputs and dispatches to `PalmierClient`.

- **[`Sources/PalmierPro/Agent/Tools/ToolDefinitions.swift`](https://github.com/palmier-io/palmier-pro/blob/main/Sources/PalmierPro/Agent/Tools/ToolDefinitions.swift)** – Declares remote tool specifications including `name`, `description`, `inputs`, and `cost` metadata.

- **[`Sources/PalmierPro/Agent/Clients/PalmierClient.swift`](https://github.com/palmier-io/palmier-pro/blob/main/Sources/PalmierPro/Agent/Clients/PalmierClient.swift)** – Low-level MCP client handling request serialization and response parsing.

- **`Sources/PalmierPro/Editor/ViewModel/EditorViewModel+Cost.swift`** – Persists generation logs and enforces subscription limits through `AccountService.shared.isSignedIn`.

## Adding a Custom AI Capability: Step-by-Step Example

To add **AI Stabilization** for shaky footage, extend the existing pipeline:

**Step 1: Define the remote tool** in [`ToolDefinitions.swift`](https://github.com/palmier-io/palmier-pro/blob/main/ToolDefinitions.swift):

```swift
// Inside Sources/PalmierPro/Agent/Tools/ToolDefinitions.swift
static let stabilize = ToolDefinition(
    name: "stabilize_video",
    description: "Stabilizes shaky video footage using AI. Returns a placeholder asset ID.",
    inputs: [.media, .optionalPrompt],
    cost: .perSecond(0.03)
)

```

**Step 2: Create the UI entry** in [`AIEditMenu.swift`](https://github.com/palmier-io/palmier-pro/blob/main/AIEditMenu.swift):

```swift
if availableActions.contains(.stabilize) {
    Button("Stabilize") { stabilize() }
}

```

**Step 3: Update EditAction availability** to include `.stabilize` for video assets in the enum definition.

**Step 4: Implement the submitter** in [`EditSubmitter.swift`](https://github.com/palmier-io/palmier-pro/blob/main/EditSubmitter.swift):

```swift
static func submitStabilize(asset: MediaAsset,
                            editor: EditorViewModel) -> Bool {
    let input = GenerationInput(
        model: ToolDefinitions.stabilize.name,
        mediaRef: asset.id,
        prompt: nil
    )
    return PalmierClient.shared.send(input, editor: editor)
}

```

**Step 5: Handle completion** using the existing `clipReplacementHandlers` infrastructure, which automatically replaces the clip once the placeholder resolves.

## Handling Asynchronous Completion and Clip Replacement

The `EditorViewModel` uses a closure-based completion system to handle asynchronous AI results. In `EditorViewModel+AIEdit.swift`, the `clipReplacementHandlers` method returns paired completion and failure closures:

```swift
private func clipReplacementHandlers(clipId: String,
                                    resetTrim: Bool)
    -> (onComplete: (@MainActor (MediaAsset) -> Void)?,
        onFailure: (@MainActor () -> Void)?) {
    markPendingReplacement(clipId: clipId)
    let fired = FirstOnlyFlag()
    let onComplete: @MainActor (MediaAsset) -> Void = { [weak self] newAsset in
        guard fired.fire() else { return }
        self?.replaceClipMediaRef(clipId: clipId,
                                  newAssetId: newAsset.id,
                                  resetTrim: resetTrim)
        self?.clearPendingReplacement(clipId: clipId)
    }
    return (onComplete, nil)
}

```

The `FirstOnlyFlag` ensures the replacement logic executes exactly once, preventing race conditions if the agent sends multiple status updates. The `resetTrim` parameter clears any in-out points when replacing source media, ensuring the new stabilized footage displays fully.

## Summary

- **AI actions are declarative** – The UI layer only decides *what* to do; `EditSubmitter` creates a `GenerationInput`, and the remote agent handles execution.

- **State is centralized** in `EditorViewModel`; pending operations, trim handling, and cost logging live in dedicated extensions, making new features additive rather than invasive.

- **Extending the pipeline** requires three steps: (1) a new `ToolDefinition` in [`ToolDefinitions.swift`](https://github.com/palmier-io/palmier-pro/blob/main/ToolDefinitions.swift), (2) a UI entry in [`AIEditMenu.swift`](https://github.com/palmier-io/palmier-pro/blob/main/AIEditMenu.swift) or [`AIEditTab.swift`](https://github.com/palmier-io/palmier-pro/blob/main/AIEditTab.swift), and (3) a thin wrapper in `EditSubmitter`.

- **Asynchronous by design** – Placeholder assets keep the UI responsive while `clipReplacementHandlers` manage the final swap when remote jobs complete.

- **Security and billing** – Every tool includes cost metadata; `EditorViewModel+Cost.swift` persists logs for usage tracking and subscription enforcement.

## Frequently Asked Questions

### How does Palmier Pro handle AI generation latency without blocking the UI?

Palmier Pro implements an **asynchronous placeholder pattern**. When a user triggers an AI action, `PalmierClient` immediately returns a placeholder asset ID while dispatching the actual work to the remote AI agent. The ViewModel sets `showGenerationPanel = true` and updates [`ProjectActivityView.swift`](https://github.com/palmier-io/palmier-pro/blob/main/ProjectActivityView.swift) with a spinner. The editor remains fully interactive; when the agent completes generation, `clipReplacementHandlers` automatically swaps the placeholder for the final media file.

### What is the role of the MCP server in Palmier Pro's AI architecture?

The **Model Context Protocol (MCP) server** acts as the bridge between the native macOS application and remote AI providers like Gemini or ElevenLabs. [`PalmierClient.swift`](https://github.com/palmier-io/palmier-pro/blob/main/PalmierClient.swift) serializes `GenerationInput` objects into MCP requests, while [`ToolDefinitions.swift`](https://github.com/palmier-io/palmier-pro/blob/main/ToolDefinitions.swift) declares the available tool schemas. This abstraction allows the video editor to support multiple AI providers through a unified interface without changing the UI or ViewModel layers.

### How do I add a new AI model to the existing pipeline?

Adding a new model requires updating [`ToolDefinitions.swift`](https://github.com/palmier-io/palmier-pro/blob/main/ToolDefinitions.swift) with a new `ToolDefinition` specifying the model name, required inputs, and cost structure. Then add a menu item in [`AIEditMenu.swift`](https://github.com/palmier-io/palmier-pro/blob/main/AIEditMenu.swift) and a corresponding submission method in [`EditSubmitter.swift`](https://github.com/palmier-io/palmier-pro/blob/main/EditSubmitter.swift) that constructs the appropriate `GenerationInput`. The existing `clipReplacementHandlers` in `EditorViewModel+AIEdit.swift` handle completion automatically, so no additional completion logic is necessary unless the feature requires custom UI feedback.

### Where does Palmier Pro store generation history and cost tracking?

Generation history persists in [`generation-log.json`](https://github.com/palmier-io/palmier-pro/blob/main/generation-log.json) within the project bundle, managed by `Sources/PalmierPro/Editor/ViewModel/EditorViewModel+Cost.swift`. This file records each AI operation including tool usage, token costs, and timestamps. The ViewModel exposes this data to the UI for project activity displays and integrates with `AccountService.shared.isSignedIn` to enforce subscription limits and usage quotas.