Audio Beat Detection and BeatStore in Palmier Pro: Core ML Implementation Guide

Palmier Pro analyzes audio assets to detect musical beats and downbeats using a Core ML pipeline, caching results in BeatStore to prevent redundant processing and enable real-time timeline snapping.

Palmier Pro is a professional video editing application that provides precise audio beat detection to synchronize edits with musical rhythm. According to the palmier-io/palmier-pro source code, the system combines a singleton BeatDetector for ML inference with a @MainActor-bound BeatStore for intelligent caching. This architecture ensures that beat detection runs efficiently without blocking the UI while maintaining data consistency across the editor.

The Beat Detection Pipeline

The beat detection workflow resides entirely in BeatDetector.swift, implemented as a singleton (BeatDetector.shared) that manages concurrent access via an AsyncSemaphore named pipelineGate, limiting execution to two simultaneous detections.

Audio Decoding and Resampling

The process begins with decodeAudio(from:), which uses AVAssetReader to extract a mono, 32-bit float PCM stream at 22.05 kHz. The method returns raw samples as a flat [Float] array optimized for the Core ML model's expected input format.

Chunked Inference with Overlapping Windows

The ML model processes audio in fixed-size chunks defined by chunkFrames = 1500. The predictLogits(samples:model:) method slides an overlapping window across the sample buffer, padding the start with six frames to ensure the first interior frame contains valid context. For each chunk, the Core ML model outputs per-frame logits representing the probability of beat and downbeat presence.

Logit Aggregation and Peak Picking

For every interior frame, the system keeps the highest-confidence logit values from overlapping chunks using a keep_first strategy. This produces two aligned time-series (beat and downbeat) at 20 ms resolution. The pickPeaks(_:) method converts these logits to probabilities via a sigmoid function, then extracts local maxima above a 0.5 threshold to identify actual beat timestamps in seconds.

BPM Estimation and Result Struct

After peak detection, estimateBPM calculates the median interval between successive beats to determine the approximate tempo. The final result wraps into a BeatAnalysis value type:

struct BeatAnalysis: Codable, Sendable, Equatable {
    let bpm: Double            // 0 when indeterminate
    let beats: [Double]        // seconds in source media
    let downbeats: [Double]
}

BeatStore Caching Architecture

While BeatDetector handles the computational heavy lifting, BeatStore.swift provides an in-memory cache and task coordination layer marked with @MainActor. This class prevents redundant analysis when multiple UI components request beat data for the same asset.

Cache Structure and File Tagging

BeatStore maintains three primary dictionaries:

• analyses: Maps media-reference strings to their cached BeatAnalysis instances
• fileTags: Stores lightweight "size + modification time" fingerprints to detect source file changes
• tasks and hydrationTasks: Track in-flight detection or cache-hydration jobs to enable work sharing

The fileTag validation ensures that modifications to the underlying media trigger fresh analysis rather than returning stale cached data.

Hydration vs. Fresh Detection

The store exposes distinct methods for different access patterns:

• analysis(for:): Returns a cached BeatAnalysis synchronously if available
• hydrate(for:): Asynchronously loads a previously saved analysis from DiskCache without invoking the ML detector
• detect(for:force:): Starts a detection task (or returns an existing one) and stores the result. The force parameter bypasses file-tag guards to trigger re-analysis

Concurrency and Task Deduplication

When detect(for:) receives multiple requests for the same asset before the first completes, the store returns the existing task handle rather than spawning duplicate work. The invalidate(_:) method clears cached data and cancels pending tasks for a specific media reference, while reset() empties the entire store—typically called when closing the editor or opening a new project.

Timeline Integration and UI

The beat data flows into the user interface through MediaVisualCache.swift, which holds the BeatStore instance and exposes analysis(for:) to the rest of the editor.

Triggering Detection from the UI

When users select Detect Beats from the timeline context menu (defined in TimelineView+BeatsMenu.swift), the editor calls mediaVisualCache.beats.detect(for:asset, force:). This either returns a cached result or initiates the full ML pipeline via BeatDetector.analysis(for:mediaRef:force:).

Upon completion, BeatStore executes the onBeatsReady closure to notify the timeline view, which triggers a repaint of beat ticks.

Beat Snapping and Visual Rendering

The timeline visualization draws beat markers using ClipRenderer.swift, specifically in drawBeatTicks, which reads the cached BeatAnalysis. For editing operations, SnapEngine.swift consumes beat timestamps through the beatSnapFrames(for:) extension method (defined on EditorViewModel), converting beat seconds into frame numbers based on the clip's FPS for precise beat-aligned snapping.

Implementation Examples

Starting a detection or reusing an existing task:

let asset: MediaAsset = // the audio asset to analyze
let beatTask = editor.mediaVisualCache.beats.detect(for: asset)

Task {
    do {
        let analysis = try await beatTask.value
        print("BPM:", analysis.bpm)
        print("Beats at:", analysis.beats)
    } catch {
        print("Beat detection failed:", error)
    }
}

Loading a previously saved analysis without running the detector:

if let hydrateTask = editor.mediaVisualCache.beats.hydrate(for: asset) {
    // Hydration runs in the background; the store calls onBeatsReady when complete
}

Getting snap frames for timeline alignment:

let frames = editor.beatSnapFrames(for: clip)   // Array of Int frame numbers

Summary

• Audio Beat Detection in Palmier Pro processes 22.05 kHz mono PCM through a chunked Core ML pipeline that outputs beat and downbeat timestamps with 20 ms precision.
• BeatDetector.swift handles the complete ML workflow—from decoding to peak picking—while limiting concurrency to two simultaneous analyses via AsyncSemaphore.
• BeatStore.swift provides a @MainActor-bound cache that deduplicates requests, validates file integrity through modification tags, and distinguishes between hydrating cached results and running fresh detection.
• The system integrates with the timeline through TimelineView+BeatsMenu.swift for UI triggers, ClipRenderer.swift for visual beat ticks, and SnapEngine.swift for frame-accurate beat snapping.

Frequently Asked Questions

How does Palmier Pro prevent redundant beat detection for the same audio file?

The BeatStore class tracks in-flight tasks using the tasks dictionary, ensuring that multiple simultaneous requests for the same asset receive the same Task handle rather than spawning duplicate ML inference jobs. Once complete, results reside in the analyses dictionary for synchronous access.

What triggers a fresh beat analysis instead of using cached results?

The detect(for:force:) method checks the fileTag dictionary to compare the asset's current size and modification time against stored metadata. If the file has changed or the force parameter is true, the system bypasses the cache and runs the full detection pipeline.

How does the beat detection handle concurrent processing without blocking the UI?

BeatDetector uses an AsyncSemaphore named pipelineGate to limit active detections to two concurrent operations, while BeatStore runs on @MainActor but delegates heavy work to background tasks. The onBeatsReady callback notifies the UI asynchronously when results become available.

What audio format does the beat detection pipeline require?

The decodeAudio(from:) method in BeatDetector.swift forces audio to mono 32-bit float PCM at 22.05 kHz using AVAssetReader, regardless of the source format, ensuring compatibility with the Core ML model's expected input dimensions.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}