Managing Multi-Cam Clip Synchronization and Angle Switching in Palmier Pro

TLDR: Palmier Pro manages multi-cam workflows through the MulticamSource model, which syncs members via audio correlation and enables non-destructive angle switching via MulticamEngine.apply, supporting both full-frame cuts and picture-in-picture layouts.

Palmier Pro provides a comprehensive framework for multi-cam clip synchronization and angle switching that treats multi-camera shoots as logical MulticamSource groups. According to the palmier-io/palmier-pro source code, the architecture separates timeline orchestration (handled by EditorViewModel+Multicam.swift) from low-level clip surgery (executed by MulticamEngine). This design enables automatic audio-based synchronization while providing frame-accurate angle replacement and overlay placement.

Core Data Model – MulticamSource

The foundation rests in Sources/PalmierPro/Models/MulticamSource.swift, where a MulticamSource represents a logical collection of video and audio members sharing a common timeline.

Each Member contains:

• mediaRef: Reference to the source media file
• kind: Classification as .angle, .mic, or .both
• angleLabel: Unique identifier for UI selection
• sync: A SyncMap containing offsetSeconds, confidence score, and locked status

Computed helpers like angles, mics, and master provide filtered access to group contents, while offsetFrames(fps:) converts temporal offsets into frame-accurate edit units.

Audio-Based Synchronization Workflow

Synchronization occurs in EditorViewModel+Multicam.swift (lines 23-44) via the syncMulticamMembers method. The process correlates audio envelopes to calculate temporal alignment:

1. Collect specifications as MulticamMemberSpec objects containing media references and optional pinned offsets.
2. Extract audio envelopes for the master reference and all pending members using AudioEnvelopeExtractor.
3. Correlate waveforms via AudioSyncCorrelator to determine offset against the master and previously anchored members.
4. Build SyncMaps with confidence scores and rebase offsets so the earliest usable point becomes zero.

let outcome = try await vm.syncMulticamMembers(
    specs: specs,
    masterRef: masterMediaRef
)

The resulting MulticamSyncOutcome contains maps consumed during group creation.

Creating a Multicam Group on the Timeline

Once synchronized, createMulticamGroup (lines 57-53 in EditorViewModel+Multicam.swift) constructs the timeline representation:

let (groupId, clipIds) = try vm.createMulticamGroup(
    specs: specs,
    syncMaps: outcome.maps,
    masterRef: masterMediaRef,
    name: "Interview"
)

This method calculates group coverage from the earliest start to latest end across all video members, then generates a program track for video and individual tracks for each microphone. The makeMemberClip function applies sync offsets via trimStartFrame and trimEndFrame to ensure frame-accurate alignment.

Switching Angles with MulticamEngine

Angle changes are non-destructive operations handled by MulticamEngine in Sources/PalmierPro/Timeline/MulticamEngine.swift (lines 29-101, 124-190, 222-274).

The AngleSwitchRequest Structure

The view model constructs AngleSwitchRequest objects specifying the frame range and target angles:

let request = AngleSwitchRequest(
    range: 1200..<1800,
    angles: ["wide-angle"],
    layout: .full
)

This transforms into a MulticamEngine.Entry defining:

• range: Affected frame indices
• slots: Target members (slot 0 = primary angle)
• layout: VideoLayout configuration for compositing

Program Track Surgery and Overlay Placement

The apply method executes six surgical steps on the timeline:

1. Locate the program track via programTrackId.
2. Clamp the requested range to actual clip coverage using clampToCoverage.
3. Clear conflicting existing overlays with clearOverlays.
4. Place new overlay clips for secondary angles via placeOverlay.
5. Split and rewrite the program track at range bounds, replacing content with the new primary member.
6. Merge adjacent through-edits using joinThroughEdits to maintain clean join points.

The engine returns a summary containing switched, merged, applied, clamped, skipped counts, and overlayClipIds.

The view model entry point switchMulticamAngles (lines 44-75) wraps this logic and handles timeline swapping.

Layout-Based Switching and Picture-in-Picture

For composite layouts, applyMulticamLayout (lines 77-88) builds requests that preserve the current primary angle while filling additional slots:

vm.applyMulticamLayout(
    clipId: someClip.id,
    layout: .twoByTwo
)

The placement closure calculates transform and crop for each overlay slot, defaulting to:

placement: { [self] clip, rect in layoutPlacement(for: clip, in: rect, fit: .fill) }

This enables simultaneous display of multiple angles within configurable LayoutRect boundaries.

Practical Implementation Examples

To switch a specific segment programmatically:

guard let groupId = vm.multicamGroup(of: selectedClip)?.id else { return }
let range = 2400..<3000
vm.switchMulticamRange(groupId: groupId, range: range, angle: "close-up")

This calls switchOrToast internally, which delegates to switchMulticamAngles and ultimately MulticamEngine.apply.

Summary

• MulticamSource provides the logical grouping with per-member SyncMap alignment data stored in MulticamSource.swift.
• Audio correlation via syncMulticamMembers establishes temporal alignment with confidence scoring.
• Group creation places trimmed clips on program and audio tracks respecting synchronized offsets.
• Angle switching operates through MulticamEngine.apply in MulticamEngine.swift, performing non-destructive timeline surgery including overlay placement and through-edit merging.
• Layout compositing uses AngleSwitchRequest with VideoLayout to populate picture-in-picture slots while maintaining the primary angle.

Frequently Asked Questions

How does Palmier Pro calculate sync confidence for multi-cam clips?

The AudioSyncCorrelator computes correlation between audio envelopes of the master reference and pending members, producing a confidence score stored in the SyncMap struct alongside the offsetSeconds value. Higher correlation values indicate stronger alignment certainty.

What happens when angle switches overlap on the timeline?

MulticamEngine.apply automatically clamps requested ranges to actual clip coverage and clears conflicting overlays before placing new ones. Adjacent edits that create seamless through-edits are merged via joinThroughEdits to prevent unnecessary cut points in the program track.

Does angle switching affect the underlying source media?

No, all angle operations are non-destructive. The MulticamEngine rewrites program track fragments and manages overlay clips as references to the original MulticamSource members, leaving source files untouched while updating timeline composition.

How are audio tracks handled when switching video angles?

The system maintains separate audio tracks for each microphone (kind: .mic or .both) defined in the MulticamSource. Video angle switching via switchMulticamRange affects only the program track (video), while audio tracks remain aligned according to their individual SyncMap offsets unless explicitly modified.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}