# How Multicam Editing and Synchronization Works in Palmier Pro: A Technical Deep Dive

> Discover how Palmier Pro's multicam editing and synchronization uses audio correlation, offset mapping, and atomic groups for perfect alignment. Learn the technical deep dive.

- Repository: [Palmier/palmier-pro](https://github.com/palmier-io/palmier-pro)
- Tags: deep-dive
- Published: 2026-07-20

---

**Palmier Pro implements multicam editing and synchronization through a three-layer architecture that combines audio-based correlation for automatic syncing, frame-accurate offset mapping, and atomic group operations to ensure angle switches maintain perfect alignment across all sources.**

Palmier Pro is an open-source video editing framework designed for complex multicamera workflows. The multicam editing and synchronization system centers on three core components—the **MulticamSource** data model, the **MulticamEngine** processing layer, and the **EditorViewModel+Multicam** façade—that together enable real-time angle switching while preserving user transforms and sync relationships.

## The Three-Layer Architecture

The multicam system is organized into distinct layers that separate data modeling from execution logic:

- **`MulticamSource`** (in [`Sources/PalmierPro/Models/MulticamSource.swift`](https://github.com/palmier-io/palmier-pro/blob/main/Sources/PalmierPro/Models/MulticamSource.swift)) defines the group structure, member metadata, and sync offsets.
- **`MulticamEngine`** (in [`Sources/PalmierPro/Timeline/MulticamEngine.swift`](https://github.com/palmier-io/palmier-pro/blob/main/Sources/PalmierPro/Timeline/MulticamEngine.swift)) performs clip rewriting, overlay management, and fragment merging.
- **`EditorViewModel+Multicam`** (in `Sources/PalmierPro/Editor/ViewModel/EditorViewModel+Multicam.swift`) provides the high-level API for group creation, synchronization, and angle switching.

This separation ensures that all angle-switching logic funnels through a single engine, guaranteeing consistent behavior across the UI, Agent tools, and undo systems.

## The Data Model: MulticamSource and Member Sync

At the heart of the system lies the **MulticamSource** structure, which groups **Members** representing camera angles, microphones, or combined video-audio tracks.

```swift
struct MulticamSource: Codable, Sendable, Equatable, Identifiable {
    struct Member: Codable, Sendable, Equatable, Identifiable {
        var id: String = UUID().uuidString
        var mediaRef: String                 // Asset identifier
        var kind: MemberKind                 // .angle, .mic, or .both
        var angleLabel: String               // Human-readable label
        var sync: SyncMap = SyncMap()        // Offset, confidence, lock flag
        var providesVideo: Bool { kind != .mic }
        var providesAudio: Bool { kind != .angle }
        var usable: Bool { sync.confidence > 0 || sync.locked }
    }
    // ...
}

```

**Sync offsets** stored in `SyncMap.offsetSeconds` express how many seconds a member is shifted relative to the group’s master clock. The `usable` property ensures only members with valid correlation confidence or manually locked offsets participate in editing. Helper methods like `coverage()`, `offsetFrames()`, and `anchorFrame()` convert between seconds and frame numbers given the timeline’s FPS.

## Creating Multicam Groups

Groups are instantiated via `EditorViewModel.createMulticamGroup`, which orchestrates the setup:

1. **Builds members** from supplied specs and sync maps.
2. **Determines program boundaries** by merging the coverage of all video members.
3. **Places a program track** (video) and optional audio tracks for each microphone.
4. **Generates clips** for each member, applying trim offsets and default fit transforms.
5. **Registers undo metadata** enabling the entire group to be removed in a single undo step.

```swift
let (groupId, _) = try editor.createMulticamGroup(
    specs: [.init(mediaRef:"camA", kind:.angle, angleLabel:"cam-a"),
            .init(mediaRef:"camB", kind:.angle, angleLabel:"cam-b"),
            .init(mediaRef:"mic1", kind:.mic,   angleLabel:"mic-1")],
    syncMaps: ["camA": .init(offsetSeconds:0, confidence:1),
               "camB": .init(offsetSeconds:5, confidence:0.9),
               "mic1": .init(offsetSeconds:2, confidence:1)],
    masterRef: "mic1",
    name: "MC",
    startFrame: 0)

```

The implementation uses `multicamSourceDurations` to fetch source lengths and `makeMemberClip` to construct each clip. The test `createLaysStampedClips` verifies that program clips and microphone offsets are correctly laid out on the timeline.

## Synchronizing Members with Audio Correlation

Automatic synchronization is performed asynchronously by `syncMulticamMembers`. The process extracts audio envelopes for each asset, then employs an **AudioSyncCorrelator** to compute the optimal lag between the master and target sources within a configurable search window. When audio is unavailable, the system falls back to time-code alignment. Finally, it **rebases** all offsets so the earliest usable member sits at zero seconds, simplifying subsequent calculations.

```swift
let outcome = try await editor.syncMulticamMembers(
    specs: specs,
    masterRef: "mic1")

```

The correlation respects minimum confidence thresholds and handles offset calculations that persist through angle switches. As demonstrated in the test `switchRewritesTrimThroughSyncMaps`, a sync offset of 5 seconds for *cam-b* means a switch at frame 600 produces a trim of 450 frames (accounting for the 5-second offset at 30 FPS).

## Switching Angles via MulticamEngine

Angle switches are handled by `MulticamEngine.apply`, which processes an `AngleSwitchRequest` specifying the target frame range and desired angles. The engine performs several critical operations:

- **Program track lookup**: Identifies the video track containing the group’s program clips via `programTrackId`.
- **Fragment enumeration**: For each program fragment intersecting the request range, calculates the valid sub-range using `clampToCoverage`.
- **Overlay management**: Removes conflicting overlays with `clearOverlays`, then inserts additional angle clips as needed via `placeOverlay`.
- **Clip rewriting**: Calls `MulticamEngine.rewrite` to swap media references and adjust trim based on stored sync offsets.
- **Transform preservation**: Applies custom placement functions for layouts or falls back to default fit transforms.
- **Fragment merging**: Collapses adjacent clips forming "through edits" using `joinThroughEdits`.

```swift
let outcome = try editor.switchMulticamAngles(
    groupId: groupId,
    requests: [.init(range: 600..<1200, angle: "cam-b")])

```

The `Outcome` structure records statistics—including `switched`, `merged`, and `clamped` counts—for UI feedback. The test `switchClampsToAngleCoverage` validates that requests outside a member’s coverage are automatically clamped and reported.

## Layout-Based Overlays and Picture-in-Picture

When applying multicam layouts such as picture-in-picture, the engine may place extra angle clips as overlays. The `VideoLayout` structure defines slots that determine where each angle appears. For every additional slot, `placeOverlay` creates a new `Clip`, computes its start frame using `anchorFrame` and the member’s offset, and inserts it in the highest-available video track above the program track.

```swift
editor.applyMulticamLayout(clipId: someClip.id, layout: .pip)

```

Custom transforms survive angle switches, while unframed fragments revert to default fits, as verified by the test `userFramingSurvivesAngleSwitch`.

## Guardrails and Atomic Operations

Multicam editing maintains strict **atomic relationships** to prevent timeline corruption:

- **Ripple edit prevention**: Operations that would split a group without shifting linked members are refused via `multicamMoveViolation`.
- **Partial edit blocking**: Partial moves or speed changes on stamped program clips are ignored to keep the group synchronized.
- **Atomic undo/redo**: Undo operations work at the group level, not per-clip. The `undo.perform("Create Multicam")` block stores both clips and group metadata, while `ungroupMulticam` removes stamps while preserving underlying media.

These constraints are enforced by tests including `partialRippleAcrossGroupRefuses`, `partialMoveRefused`, and `speedRefusedOnStampedClips`.

## Reading the Program Timeline

The UI can display which angle is active over time using `multicamProgramRows`. This function walks the program clips, merges consecutive fragments with identical angles, and returns a run-length encoded table.

```swift
let rows = editor.multicamProgramRows(groupId: groupId)
// Returns: [["cam-a", 0, 600], ["cam-b", 600, 1200], ["cam-a", 1200, 3600]]

```

The `programRowsRunLengthMerge` test validates this compression behavior, enabling efficient timeline visualization without processing every individual frame.

## Summary

- **Single source of truth**: All angle-switching logic routes through `MulticamEngine.apply`, ensuring consistency across UI, Agent tools, and undo layers.
- **Sync-first workflow**: Offsets are computed once via audio correlation and stored in `SyncMap`; subsequent edits adjust trim frames using these persistent offsets.
- **Atomic guarantees**: Guardrails prevent partial edits that would desynchronize the group, preserving timeline integrity during complex operations.
- **Extensible layout system**: New `VideoLayout` configurations automatically trigger overlay insertion through the existing engine methods without requiring structural changes.

## Frequently Asked Questions

### How does Palmier Pro calculate sync offsets between camera angles?

Palmier Pro calculates sync offsets through the `syncMulticamMembers` method, which extracts audio envelopes and uses an `AudioSyncCorrelator` to compute the optimal lag between the master source and each target. The system respects a configurable search window and minimum confidence threshold, falling back to time-code alignment when audio is unavailable. All offsets are then rebased so the earliest usable member starts at zero seconds.

### What happens when you switch angles outside a member's coverage range?

When an angle switch request extends beyond a member's actual media coverage, the `MulticamEngine` automatically clamps the operation to the valid range. The `switchClampsToAngleCoverage` test verifies this behavior, and the `Outcome` structure reports which portions were clamped so the UI can provide feedback to the user.

### How does the system handle undo operations for multicam groups?

Undo operations in Palmier Pro work at the group level rather than per-clip. The `createMulticamGroup` method registers an undo block that stores both the created clips and the group metadata, allowing the entire multicam structure to be removed or restored as a single atomic unit. This prevents the timeline from entering inconsistent states where some group members are deleted while others remain.

### Can custom video transforms survive angle switches in Palmier Pro?

Yes, custom transforms and crops applied by users survive angle switches. When `switchMulticamAngles` processes a request, it preserves any user-applied framing on the program track. If a custom layout is applied, the engine uses a placement function to maintain these transforms; otherwise, it falls back to the default fit transform. The test `userFramingSurvivesAngleSwitch` explicitly confirms this preservation behavior.