Implementing AI Transition Placement with Visual Reasoning in Palmier Pro

Palmier Pro handles AI transition placement by inserting placeholder assets into validated timeline gaps and replacing them with AI-generated media once visual seed frames are processed and rendering completes.

Palmier Pro's AI transition placement with visual reasoning enables editors to automatically generate seamless video transitions by analyzing visual context from adjacent clips. The system validates timeline gaps, captures reference frames for AI processing, and manages placeholder assets with full undo support. This implementation in the palmier-io/palmier-pro repository ensures generated content fits precisely within empty gaps while maintaining responsive UI feedback through Swift concurrency.

Gap Validation and Placement Creation

The workflow begins when a user selects an empty gap in the timeline. The EditorViewModel.aiTransitionAvailability(for:) method validates that the gap resides on a video track, contains a non-zero duration, and does not exceed the hard-coded limit of 15 seconds (Sources/PalmierPro/Editor/ViewModel/EditorViewModel+AITransition.swift lines 21-30).

Once validated, beginAITransition(gap:) constructs a PendingTransitionPlacement struct that stores critical metadata: the active timeline ID, track index, start frame, and gap length in frames. This placement object serves as the authoritative reference throughout the generation lifecycle.

Visual Seed Generation Process

After placement creation, a background Task named transitionSeedTask executes off the main actor to prevent UI blocking. This task calls captureFrameToMedia to extract the first and last frame media assets from the gap boundaries, then opens the generation panel using these frames as reference images via seedGenerationPanel.

The task periodically invokes transitionSeedIsCurrent(_:) to verify the operation remains valid. If the gap becomes filled or the active timeline changes (comparing activeTimelineId against placement.timelineId or detecting transitionGapIsEmpty), the task aborts promptly using Task.checkCancellation (Sources/PalmierPro/Editor/ViewModel/EditorViewModel+AITransition.swift lines 45-63).

Placeholder Insertion and Gap Safety

While the seed generation runs, the UI inserts a placeholder clip using placeGeneratingTransitionClip(placeholderId:placement:). This method first confirms the gap remains empty through transitionSeedIsCurrent, then registers an undoable insertion via undo.withoutRegistration followed by placeClip.

If the gap becomes occupied during the check, the method returns nil and displays a toast notification through refuseWithToast, preventing invalid placements (Sources/PalmierPro/Editor/ViewModel/EditorViewModel+AITransition.swift lines 87-104).

Finalizing AI Transition Clips

When the AI model finishes rendering, finalizeTransitionClip(placeholderId:asset:) replaces the placeholder with the generated media. The method computes the real frame count from the asset duration and adjusts the clip speed to ensure exact gap filling using the formula:

clip.speed = Double(realFrames) / Double(max(1, clip.durationFrames))

It also clears any existing trimming by setting clip.trimStartFrame = 0. This operation is recorded in the undo stack, allowing users to revert the entire AI transition with a single undo action (Sources/PalmierPro/Editor/ViewModel/EditorViewModel+AITransition.swift lines 13-20).

Visual Reasoning in the Timeline UI

Visual reasoning integrates directly into the timeline view through context menu entries and hit-testing logic. The "Create AI Transition" menu item appears under the gap's edit menu via performCreateAITransition(_:) (Sources/PalmierPro/Timeline/TimelineView+AIEditMenu.swift lines 11-18).

The system uses hitTestGap(at:trackIndex:geometry:) to determine cursor position over valid gaps, surfacing the AI transition action only when appropriate empty space exists (Tests/PalmierProTests/Timeline/AITransitionPlacementTests.swift lines 54-68).

Implementation Example

The following Swift code demonstrates the complete workflow from gap validation through finalization:

// 1️⃣ Validate gap and start AI transition
let gap = GapSelection(trackIndex: 0, range: FrameRange(start: 100, end: 160))
editor.beginAITransition(gap: gap)   // calls aiTransitionAvailability → beginAITransition

// 2️⃣ Insert placeholder while the seed runs
let placement = PendingTransitionPlacement(
    timelineId: editor.activeTimelineId,
    trackIndex: 0,
    gapStartFrame: 100,
    gapLengthFrames: 60
)
editor.placeGeneratingTransitionClip(placeholderId: "gen-1", placement: placement)

// 3️⃣ When the AI model finishes, replace placeholder with real media
let generatedAsset = MediaAsset(id: "gen-1",
                                url: URL(fileURLWithPath: "/tmp/generated.mp4"),
                                type: .video,
                                name: "generated",
                                duration: 4.0)   // 4 seconds
editor.finalizeTransitionClip(placeholderId: "gen-1", asset: generatedAsset)

Key Source Files

File	Role
Sources/PalmierPro/Editor/ViewModel/EditorViewModel+AITransition.swift	Core logic for gap validation, seed creation, placeholder insertion, and finalization
Sources/PalmierPro/Timeline/TimelineView+AIEditMenu.swift	Adds the "Create AI Transition" menu item and handles command dispatch
Sources/PalmierPro/Timeline/TimelineView.swift	Implements hit-testing logic to detect gaps under the cursor
Tests/PalmierProTests/Timeline/AITransitionPlacementTests.swift	Verifies placement, undo behavior, stale-seed handling, and duration snapping

Summary

• Gap Validation: AI transitions are only created for video track gaps under 15 seconds with valid duration.
• Visual Context: The system captures adjacent frame media as seeds for AI generation, running asynchronously off the main actor.
• Safe Placement: Placeholder insertion includes race-condition checks to prevent filling occupied gaps.
• Undo Integration: Both placeholder insertion and final retiming are undoable operations wrapped in proper undo registrations.
• UI Integration: Visual reasoning is surfaced through timeline hit-testing and context menus in the SwiftUI interface.

Frequently Asked Questions

What is the maximum gap duration supported for AI transitions in Palmier Pro?

Palmier Pro enforces a hard-coded limit of 15 seconds for AI transition gaps. The aiTransitionAvailability(for:) method explicitly checks that the selected gap duration does not exceed this threshold before allowing placement creation.

How does the system handle gaps that become occupied during AI generation?

The transitionSeedTask periodically calls transitionSeedIsCurrent(_:) to validate that the gap remains empty and belongs to the active timeline. If the gap fills or the timeline changes, the task detects this state and aborts via Task.checkCancellation, while placeGeneratingTransitionClip performs a final check and shows a toast notification if the gap is no longer available.

Can users undo an AI transition after it has been finalized?

Yes, both the placeholder insertion and the final media replacement are registered as undoable operations. The finalizeTransitionClip method records its retiming actions in the undo stack, allowing users to revert the entire AI transition with a single undo command.

How does Palmier Pro capture visual context for AI transition generation?

The system extracts visual reference frames using captureFrameToMedia to grab the first and last frames adjacent to the gap. These frames are passed to seedGenerationPanel as reference images, providing the AI model with visual context to generate seamless transitions between the existing clips.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}