# How to Implement Custom AVFoundation Video Compositors for Effect Chaining in Palmier Pro > Learn to implement custom AVFoundation video compositors in Palmier Pro. Chain visual effects by subclassing and populating the effects array for advanced frame processing. Master video compositing now. - Repository: [Palmier/palmier-pro](https://github.com/palmier-io/palmier-pro) - Tags: how-to-guide - Published: 2026-07-27 --- **To implement custom AVFoundation video compositors in Palmier Pro, subclass NSObject and conform to AVVideoCompositing—using CustomVideoCompositor as your render engine—then chain visual effects by populating the effects array inside CompositorInstruction objects that drive the frame processing pipeline.** Palmier Pro extends AVFoundation's native rendering capabilities through a modular compositing architecture defined in the palmier-io/palmier-pro repository. The system centers on a **custom AVFoundation video compositor** implemented in [`Sources/PalmierPro/Compositing/CustomVideoCompositor.swift`](https://github.com/palmier-io/palmier-pro/blob/main/Sources/PalmierPro/Compositing/CustomVideoCompositor.swift), which processes pixel buffers while executing ordered sequences of VideoEffect objects defined in `CompositorInstruction` structs. This design enables you to construct reusable, chainable visual effects that integrate seamlessly with standard AVPlayer and export workflows. ## CustomVideoCompositor Implementation Details The `CustomVideoCompositor` class serves as the primary entry point for custom rendering. Defined in [`Sources/PalmierPro/Compositing/CustomVideoCompositor.swift`](https://github.com/palmier-io/palmier-pro/blob/main/Sources/PalmierPro/Compositing/CustomVideoCompositor.swift), it adopts the `AVVideoCompositing` protocol and manages the lifecycle of composition requests. ### Class Structure and Protocol Conformance The class declaration includes conformance to `@unchecked Sendable` for thread safety across the render queue: ```swift final class CustomVideoCompositor: NSObject, AVVideoCompositing, @unchecked Sendable { var videoComposition: AVVideoComposition! var sourceTrackIDForFrameTiming: CMPersistentTrackID = kCMPersistentTrackID_Invalid var sourceSampleDataTrackIDs: [CMPersistentTrackID] = [] private var renderContext: AVVideoCompositionRenderContext! private var instructionQueue: [AVVideoCompositionInstruction] = [] private var cancelled = false } ``` Key properties include: - **`videoComposition`** – References the driving composition configuration. - **`sourceTrackIDForFrameTiming`** – Identifies the track providing timing metadata. - **`sourceSampleDataTrackIDs`** – Lists auxiliary sample data tracks available during processing. ### Handling Render Context Updates AVFoundation calls `renderContextChanged(_:)` whenever the output dimensions or pixel format changes. Store this context to configure pixel buffer pools or Metal textures for your effects: ```swift func renderContextChanged(_ newRenderContext: AVVideoCompositionRenderContext) { renderContext = newRenderContext } ``` ### Processing Requests and Managing State The `startRequest(_:)` method receives `AVVideoCompositionRequest` objects and manages an internal instruction queue: ```swift func startRequest(_ request: AVVideoCompositionRequest) { instructionQueue.append(request.videoCompositionInstruction) processNext() } ``` The private `processNext()` method retrieves the next instruction and iterates through required source tracks. In the source implementation, this method demonstrates where you would execute your effect chain: ```swift private func processNext() { guard !cancelled, let instruction = instructionQueue.first else { return } for trackID in instruction.requiredSourceTrackIDs ?? [] { let trackIDValue = trackID.int32Value // Retrieve source buffers and apply effects... } instructionQueue.removeFirst() } ``` To halt rendering, the class implements `cancelAllRequests()`: ```swift func cancelAllRequests() { cancelled = true instructionQueue.removeAll() } ``` ## Effect Chaining with CompositorInstruction The `CompositorInstruction` struct, located in [`Sources/PalmierPro/Compositing/CompositorInstruction.swift`](https://github.com/palmier-io/palmier-pro/blob/main/Sources/PalmierPro/Compositing/CompositorInstruction.swift), bridges the gap between high-level editing timelines and low-level pixel processing. ### Instruction Structure This struct conforms to `AVVideoCompositionInstruction` and stores both standard composition metadata and a custom effects array: ```swift struct CompositorInstruction: AVVideoCompositionInstruction { var timeRange: CMTimeRange var enablePostProcessing: Bool = false var requiredSourceTrackIDs: [NSValue]? // Track IDs needed for this segment var containsTweening: Bool = false // Custom properties for effect chaining var effects: [VideoEffect] = [] init(timeRange: CMTimeRange, effects: [VideoEffect] = []) { self.timeRange = timeRange self.effects = effects } } ``` The **`effects`** array determines the sequential order of transformations applied to each frame, enabling **effect chaining** where the output of one filter becomes the input for the next. ## Defining the VideoEffect Protocol Since CompositorInstruction stores `[VideoEffect]`, you must define a protocol that standardizes how individual effects transform pixel data. Each implementation receives a `CVPixelBuffer` and the render context, then returns a modified buffer: ```swift import CoreVideo protocol VideoEffect { /// Processes the input buffer and returns a transformed buffer. /// - Parameters: /// - buffer: The source CVPixelBuffer to process. /// - context: The AVVideoCompositionRenderContext providing size and timing data. /// - Returns: A new or recycled CVPixelBuffer containing the processed image, or nil on failure. func apply(to buffer: CVPixelBuffer, context: AVVideoCompositionRenderContext) -> CVPixelBuffer? } ``` ### Concrete Effect Implementation Create concrete types conforming to this protocol. For example, a Core Image-based color adjustment: ```swift import CoreImage struct ColorGradingEffect: VideoEffect { let saturation: Double let brightness: Double func apply(to buffer: CVPixelBuffer, context: AVVideoCompositionRenderContext) -> CVPixelBuffer? { let ciImage = CIImage(cvPixelBuffer: buffer) let filtered = ciImage .applyingFilter("CIColorControls", parameters: [ "inputSaturation": saturation, "inputBrightness": brightness ]) // Create output buffer matching render context dimensions var outputBuffer: CVPixelBuffer? let pixelBufferFormat = CVPixelBufferGetPixelFormatType(buffer) CVPixelBufferCreate(kCFAllocatorDefault, Int(context.size.width), Int(context.size.height), pixelBufferFormat, nil, &outputBuffer) guard let output = outputBuffer else { return nil } let ciContext = CIContext(options: nil) ciContext.render(filtered, to: output) return output } } ``` ## Integrating Text Overlays CustomVideoCompositor provides hooks for combining pixel-based effects with vector-based overlays. The class exposes an optional `CALayer` property for text or graphics: ```swift var textOverlayLayer: CALayer? func setTextOverlay(_ layer: CALayer) { textOverlayLayer = layer } ``` During frame processing, you can composite this layer onto the final buffer after executing the VideoEffect chain, facilitating workflows such as color grading followed by subtitle rendering. ## Complete Integration Example To assemble a complete custom AVFoundation video compositor workflow: ```swift import AVFoundation // 1. Define effect chain let effectChain: [VideoEffect] = [ ColorGradingEffect(saturation: 1.2, brightness: 0.1), // Additional effects... ] // 2. Create instruction for 5-second duration let instruction = CompositorInstruction( timeRange: CMTimeRange(start: .zero, duration: CMTime(seconds: 5, preferredTimescale: 600)), effects: effectChain ) // 3. Configure video composition let composition = AVMutableVideoComposition() composition.renderSize = CGSize(width: 1920, height: 1080) composition.frameDuration = CMTime(value: 1, timescale: 30) composition.customVideoCompositorClass = CustomVideoCompositor.self composition.instructions = [instruction] // 4. Apply to player or export session let playerItem = AVPlayerItem(asset: videoAsset) playerItem.videoComposition = composition ``` ## Summary - **CustomVideoCompositor** in [`Sources/PalmierPro/Compositing/CustomVideoCompositor.swift`](https://github.com/palmier-io/palmier-pro/blob/main/Sources/PalmierPro/Compositing/CustomVideoCompositor.swift) implements the `AVVideoCompositing` protocol and manages the render queue via `startRequest(_:)` and `processNext()`. - **CompositorInstruction** in [`Sources/PalmierPro/Compositing/CompositorInstruction.swift`](https://github.com/palmier-io/palmier-pro/blob/main/Sources/PalmierPro/Compositing/CompositorInstruction.swift) conforms to `AVVideoCompositionInstruction` and stores an ordered `effects` array for declarative effect chaining across specific time ranges. - The **VideoEffect** protocol defines the transformation interface, accepting `CVPixelBuffer` and `AVVideoCompositionRenderContext` to produce modified frames. - **Effect chaining** executes sequentially in the order defined in the `effects` array, with each effect receiving the buffer output from the previous stage. - **Text overlay integration** is supported through the `textOverlayLayer` property and `setTextOverlay(_:)` method, enabling hybrid pixel and vector rendering. ## Frequently Asked Questions ### How does CustomVideoCompositor handle asynchronous rendering requests? CustomVideoCompositor processes requests synchronously within `startRequest(_:)` in the provided implementation, immediately invoking `processNext()`. To implement asynchronous processing, dispatch pixel buffer operations to a background queue, then call `request.finishWithComposedVideoFrame()` upon completion. Ensure thread-safe access to the `instructionQueue` using the `@unchecked Sendable` conformance and appropriate locking mechanisms. ### What is the purpose of the requiredSourceTrackIDs property in CompositorInstruction? The `requiredSourceTrackIDs` property—as defined in the `AVVideoCompositionInstruction` protocol implementation within `CompositorInstruction`—specifies which video track IDs the compositor requires data from for a given time range. CustomVideoCompositor uses these identifiers to retrieve source frames via `request.sourceFrame(byTrackID:)`, ensuring the compositor only accesses relevant media tracks during processing. ### Can Metal shaders be integrated into the VideoEffect chain? Yes. Since the VideoEffect protocol operates on `CVPixelBuffer` inputs and outputs, you can implement effects using Metal compute shaders by creating `MTLTexture` objects from the pixel buffer, executing your shader kernel, then returning the resulting buffer. This approach mixes seamlessly with Core Image-based effects in the same `CompositorInstruction.effects` array. ### How do I integrate text overlays with pixel-based effects? Set the `textOverlayLayer` property on your CustomVideoCompositor instance using `setTextOverlay(_:)`. During frame processing inside `processNext()`, render the CALayer onto the final pixel buffer after executing the VideoEffect chain—typically by creating a Core Image or Metal representation of the layer and compositing it over the processed image.