How to Implement Timeline Keyframe Interpolation with CMTime Math in Palmier Pro

Palmier Pro converts frame-based keyframe tracks into CMTime-based ramps by mapping integer frame indices to rational time values using the timeline's FPS timescale, then approximating smooth curves through piecewise-linear subdivision.

Palmier Pro represents film timelines as discrete frame indices while rendering through AVFoundation's CMTime architecture. When animating properties like opacity, volume, or spatial transforms, the framework must bridge these two time models—storing user keyframes as simple integers while emitting precise rational-time ramps that AVFoundation can consume.

Core Data Model for Keyframes

The animation system centers on two generic types defined in Sources/PalmierPro/Models/Keyframe.swift.

The Keyframe Structure

The Keyframe<Value> struct stores a single animation value at a specific frame index along with its interpolation behavior.

// https://github.com/palmier-io/palmier-pro/blob/main/Sources/PalmierPro/Models/Keyframe.swift#L7-L24
struct Keyframe<Value: Codable & Sendable & Equatable>: Codable, Sendable, Equatable {
    var frame: Int                 // timeline frame (absolute)
    var value: Value
    var interpolationOut: Interpolation = .smooth
}

KeyframeTrack Management

KeyframeTrack<Value> maintains a sorted collection of keyframes for a single animatable property. It provides upsert, remove, and move operations that maintain temporal ordering. Each track attaches to a Clip instance, allowing independent animation of opacity, position, scale, rotation, crop, and volume.

Frame-to-CMTime Conversion

AVFoundation represents time as CMTime, a rational number structure (value / timescale). Palmier Pro derives the timescale from the timeline's configured frames-per-second (FPS), ensuring frame-accurate alignment.

The conversion happens in Sources/PalmierPro/Preview/CompositionBuilder.swift through a dedicated helper:

// https://github.com/palmier-io/palmier-pro/blob/main/Sources/PalmierPro/Preview/CompositionBuilder.swift#L582-L585
func cmTime(_ frame: Int) -> CMTime {
    CMTime(value: CMTimeValue(frame), timescale: timescale)
}

This method converts absolute frame indices into CMTime values that AVFoundation uses for composition ranges and parameter ramps.

Interpolation Algorithms

All animatable types conform to KeyframeInterpolatable, which defines how values blend between keyframes.

Linear and Hold Modes

For scalar Double values, linear interpolation applies a simple weighted blend:

// https://github.com/palmier-io/palmier-pro/blob/main/Sources/PalmierPro/Models/Keyframe.swift#L61-L64
extension Double: KeyframeInterpolatable {
    static func keyframeInterpolate(_ a: Double, _ b: Double, t: Double) -> Double {
        a + (b - a) * t
    }
}

Hold mode emits an instantaneous value change at the keyframe boundary, while composite types like AnimPair or Crop delegate interpolation to their constituent scalar components.

Smooth Interpolation with Smoothstep

Smooth mode applies the classic smoothstep curve (t * t * (3 - 2t)) to ease in and out of keyframes. Because AVFoundation only accepts linear ramps, Palmier Pro pre-computes intermediate subdivision points to approximate the curve.

Building AVFoundation Ramps

The CompositionBuilder generates concrete time ranges by processing keyframe tracks through a normalization pipeline (lines 747–866).

The workflow follows four steps:

1. Normalize keyframes to clip-relative offsets using toOffset(_:) and constrain them to the clip duration.
2. Compute offset points including keyframe frames, fade boundaries, and smooth-subdivision interior points.
3. Sample values at each offset using the appropriate interpolation mode.
4. Emit CMTimeRange objects spanning each interval to AVFoundation APIs like setVolumeRamp or setOpacityRamp.

// https://github.com/palmier-io/palmier-pro/blob/main/Sources/PalmierPro/Preview/CompositionBuilder.swift#L747-L766
let start = CMTime(value: CMTimeValue(clip.startFrame), timescale: timescale)
let end   = CMTime(value: CMTimeValue(clip.endFrame),   timescale: timescale)
params.setVolumeRamp(
    fromStartVolume: startVolume,
    toEndVolume:   endVolume,
    timeRange: CMTimeRange(start: start, end: end)
)

Handling Smooth Curves with Frame Subdivision

Since AVFoundation ramps are strictly linear, smooth interpolation requires breaking the curve into short linear segments. The smoothSubdivisions(from:to:) static method computes these interior frames:

// https://github.com/palmier-io/palmier-pro/blob/main/Sources/PalmierPro/Preview/CompositionBuilder.swift#L72-L77
static func smoothSubdivisions(from a: Int, to b: Int) -> [Int] {
    guard b > a else { return [] }
    let span = Double(b - a)
    let raw = (1..<smoothSegments).map { a + Int((span * Double($0) / Double(smoothSegments)).rounded()) }
    return Array(Set(raw)).sorted()
}

These subdivision offsets are injected into the envelope's extraOffsets set, ensuring the final animation approximates a smooth curve through a series of short linear CMTime ramps.

Practical Implementation Example

To insert a volume keyframe with smooth interpolation and render the composition:

// 1️⃣ Insert a smooth‑interpolated volume keyframe at absolute frame 120
var clip = myProject.clip(at: /* … */)
clip.upsertKeyframe(
    in: \.volumeTrack,
    frame: 120,
    value: 0.8
)
clip.setInterpolation(for: .volume, atFrame: 120, .smooth)

// 2️⃣ Build the composition – the builder will turn those frames into CMTime ramps
let composition = try CompositionBuilder.build(from: myProject)

Internally, upsertKeyframe converts the absolute frame 120 to a clip-relative offset using toOffset(_:), while CompositionBuilder later converts it back to CMTime via cmTime(_:) and subdivides the smooth segment for rendering.

Summary

• Store keyframes as generic Keyframe<Value> structs with frame indices and interpolation modes in Keyframe.swift.
• Convert frames to CMTime using rational math (CMTimeValue / timescale) based on timeline FPS in CompositionBuilder.swift.
• Subdivide smooth curves into linear segments using smoothSubdivisions(from:to:) to satisfy AVFoundation's linear ramp requirements.
• Emit ramps by sampling values at computed offsets and constructing CMTimeRange objects for setVolumeRamp and similar APIs.

Frequently Asked Questions

What is CMTime and why does Palmier Pro use it for timeline keyframe interpolation?

CMTime is Apple's Core Media structure representing time as a rational number (value / timescale). Palmier Pro uses it because AVFoundation's composition and editing APIs require CMTime values for all time ranges and parameter ramps, ensuring sample-accurate synchronization across different media with varying frame rates.

How does Palmier Pro handle smooth interpolation when AVFoundation only supports linear ramps?

Palmier Pro approximates smooth curves by subdividing the interval between keyframes into multiple short linear segments. The smoothSubdivisions(from:to:) method generates intermediate frame offsets that follow the smoothstep curve, then emits discrete linear ramps between each subdivision point, creating the illusion of smooth easing.

Why are keyframes stored as offsets rather than absolute timeline frames?

Keyframes store clip-relative offsets to maintain animation integrity when clips are moved on the timeline. The Clip extension provides toOffset(_:) and toAbs(_:) helpers to translate between absolute timeline frames (used by the UI) and internal offset values (used for storage and interpolation), ensuring keyframes move with their parent clips.

How do I convert a frame number to CMTime for custom composition building?

Use the cmTime(_:) helper pattern from CompositionBuilder.swift, constructing CMTime(value: CMTimeValue(frame), timescale: timescale) where timescale equals your timeline's FPS. This ensures frame indices map precisely to the rational time values required by AVFoundation's CMTimeRange and mixing parameters.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}