CompositionBuilder in Palmier Pro: How It Constructs AVCompositions from Timelines

CompositionBuilder is a static utility enum that transforms high-level Timeline models into fully configured AVFoundation compositions by validating inputs, inserting media clips, generating custom compositor instructions, and bundling the results into a CompositionResult ready for playback or export.

The CompositionBuilder serves as the central translation layer in the palmier-io/palmier-pro repository, bridging the gap between Palmier Pro's declarative Timeline models and AVFoundation's imperative composition APIs. Located at Sources/PalmierPro/Preview/CompositionBuilder.swift, this enum-based utility never requires instantiation and provides a purely functional interface for converting track-based editor data into renderable AVComposition objects.

The Seven-Step Construction Pipeline

The CompositionBuilder constructs AVCompositions through a deterministic pipeline that processes timeline metadata, resolves media references, and assembles layered instructions for the custom video compositor.

Step 1: Timeline Validation

Before creating any AVFoundation objects, the builder validates the input Timeline model. It checks that FPS, width, and height values are positive integers, throwing InvalidTimelineError if any constraints fail.

guard timeline.fps > 0, timeline.width > 0, timeline.height > 0 else {
    throw InvalidTimelineError()
}

Source: CompositionBuilder.swift, lines 41-44

Step 2: Creating the Mutable Composition

Upon validation, the builder instantiates an AVMutableComposition that serves as the container for all audio and video tracks. This mutable composition grows dynamically as the builder processes each timeline track.

let composition = AVMutableComposition()

Source: CompositionBuilder.swift, line 45

Step 3: Processing Timeline Tracks

The builder iterates through the timeline's track array, sorting clips chronologically and distinguishing between audio and video content. For each media clip, it calls loadSource (lines 45-88) to resolve the mediaRef string into an AVURLAsset. This resolution step handles offline media detection and converts static assets—like images or Lottie animations—into video tracks through automatic generation. The method returns a LoadOutcome enum value (loaded, offline, or unprocessable) that determines whether the clip can be inserted.

Step 4: Inserting Clips with Time Scaling

For each resolvable clip, the builder calls insertClip (lines 101-146) to map the timeline clip's local time range into the composition's global timeline. This method handles:

• Gap filling between non-contiguous clips
• Speed scaling by adjusting CMTime ranges for playback rate modifications
• Track insertion into either temporary audio tracks (for speed-altered clips) or permanent video tracks (one per timeline track)

For video tracks, the builder creates one mutable track per timeline track to maintain layer ordering.

Step 5: Adding the Black Background Layer

After inserting all media clips, the builder guarantees a visible backdrop by calling insertBlackBackground (lines 200-215). This generates a solid black video layer spanning the full render dimensions and timeline duration, ensuring that transparent areas or gaps render as black rather than showing undefined pixel data.

Step 6: Building Visual Instructions

The buildVisuals method (lines 776-822) creates the advanced composition objects required for rendering:

• AVMutableAudioMix: Configures volume envelopes, mute flags, and audio ramping
• AVVideoComposition: References the CustomVideoCompositor class and includes CompositorInstruction objects defining how layers should be blended for each time segment

The compositor instructions are generated by compositorInstructions (lines 424-486), which segments the timeline at every clip start/end boundary and assembles layer plans describing opacity, transforms, and scaling for each segment.

Step 7: Returning the CompositionResult

The pipeline culminates in a CompositionResult struct (defined lines 15-24) that bundles all generated objects:

return CompositionResult(
    composition: composition,
    audioMix: audioMix,
    videoComposition: videoComposition,
    trackMappings: mappings,
    offlineMedia: offlineSet,
    unprocessableMedia: unprocessableSet
)

Source: CompositionBuilder.swift, lines 27-38

Key Architectural Components

build – The public entry point accepting a Timeline, render size, and resolution closures. It orchestrates the seven-step pipeline and returns the populated CompositionResult. Source: lines 34-39 and 53-138

loadSource – Resolves media references to URLs, normalizes video orientation, and handles image-to-video conversion. Returns LoadOutcome to signal whether the asset is available for insertion. Source: lines 45-88

insertClip – Performs the low-level insertTimeRange calls on AVMutableCompositionTrack, handling speed scaling through CMTime mathematics and maintaining the insertion cursor for sequential placement. Source: lines 101-146

buildVisuals – Generates AVMutableAudioMix for volume control and AVVideoComposition for the custom compositor, configuring render size and frame duration based on timeline properties. Source: lines 776-822

compositorInstructions – Translates timeline clip stacks into CompositorInstruction objects that the CustomVideoCompositor interprets during rendering, handling layer ordering and time-range segmentation. Source: lines 424-486

Practical Implementation Examples

Building a Composition for Preview Playback

This pattern from VideoEngine.swift (lines 153-156) demonstrates the standard integration:

let renderSize = CGSize(width: timeline.width, height: timeline.height)

do {
    let result = try await CompositionBuilder.build(
        timeline: timeline,
        resolveURL: { ref in /* map mediaRef to local URL */ nil },
        resolveSourceSize: { _ in nil },
        renderSize: renderSize
    )

    let playerItem = AVPlayerItem(asset: result.composition)
    playerItem.audioMix = result.audioMix
    playerItem.videoComposition = result.videoComposition
    
    let player = AVPlayer(playerItem: playerItem)
    // Present player in AVPlayerViewController or SwiftUI VideoPlayer
} catch {
    print("Composition failed: \(error)")
}

Inspecting Track Mappings

The CompositionResult includes metadata mapping timeline tracks to composition tracks:

for mapping in result.trackMappings {
    switch mapping.kind {
    case .timeline(let trackIdx, let clipIds):
        print("Track \(trackIdx) contains clips: \(clipIds ?? [])")
    case .blackBackground(let timeRange):
        print("Background layer duration: \(timeRange.duration)")
    }
}

Source: TrackMapping definition and usage in CompositionBuilder.swift, lines 94-100

Configuring the Custom Video Compositor

When building the video composition manually using the result:

let videoComp = AVVideoComposition(configuration: {
    var cfg = AVVideoComposition.Configuration()
    cfg.renderSize = renderSize
    cfg.frameDuration = CMTime(value: 1, timescale: CMTimeScale(timeline.fps))
    cfg.customVideoCompositorClass = CustomVideoCompositor.self
    cfg.instructions = result.videoComposition.instructions
    return cfg
}())

Source: Implementation reference from buildVisuals, lines 808-822

Core Source Files

• Sources/PalmierPro/Preview/CompositionBuilder.swift – Complete builder implementation including track handling, source loading, and visual instruction generation
• Sources/PalmierPro/Models/Timeline.swift – Defines Timeline, Track, and Clip structures consumed by the builder
• Sources/PalmierPro/Preview/VideoEngine.swift – Production usage demonstrating preview rendering calls
• Sources/PalmierPro/Export/ExportService.swift – Export pipeline integration using the same builder for final output
• Sources/PalmierPro/Compositing/CustomVideoCompositor.swift – Metal-based compositor that interprets instructions generated by the builder

Summary

• CompositionBuilder is a static enum in CompositionBuilder.swift that converts Timeline models to AVFoundation compositions without requiring instantiation
• The construction pipeline validates inputs, creates an AVMutableComposition, iterates tracks to insert clips via insertClip, adds a mandatory black background, and generates CompositorInstruction objects
• Key methods include build (entry point), loadSource (asset resolution), and buildVisuals (audio/video mix creation)
• The builder outputs a CompositionResult containing the composition, audio mix, video composition, and metadata about offline or unprocessable media
• Text-based clips are skipped during composition construction and rendered separately by the view layer

Frequently Asked Questions

Why is CompositionBuilder implemented as an enum instead of a class?

CompositionBuilder uses an enum with no cases and only static members to enforce its stateless, functional design. This prevents accidental instantiation and makes explicit that the builder maintains no internal state—each call to build is completely independent and thread-safe, which is critical for preview and export operations that may run on background queues.

How does CompositionBuilder handle images and Lottie animations in the timeline?

Through the loadSource method (lines 45-88), the builder detects non-video media types and automatically generates video tracks from static images or Lottie JSON files. This conversion happens during the resolution phase before clip insertion, ensuring that the AVMutableComposition contains only standard video tracks that AVFoundation can process natively, with the generated assets normalized to match the timeline's render dimensions and orientation.

What is the relationship between CompositionBuilder and CustomVideoCompositor?

CompositionBuilder generates the structural AVVideoComposition and populates its instructions array with CompositorInstruction objects, but it does not perform the actual pixel-level rendering. The CustomVideoCompositor (defined in CustomVideoCompositor.swift) is the Metal-accelerated class that reads these instructions during playback or export to blend layers, apply transforms, and handle opacity. The builder prepares the "what" and "when" (layer stack per time range), while the compositor handles the "how" (pixel blending).

How does the builder handle clips with speed adjustments or gaps?

The insertClip method (lines 101-146) manages speed scaling by calculating scaled CMTime ranges—when a clip plays at 2x speed, its duration in the composition is halved. For gaps between clips, the method advances the insertion cursor without inserting media, leaving empty time ranges in the track. Speed-altered audio clips receive temporary tracks to accommodate the time-stretching, while video clips insert directly into their respective timeline tracks.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}