# Audio Beat Detection and BeatStore in Palmier Pro: Core ML Implementation Guide

> Implement audio beat detection and BeatStore in Palmier Pro using Core ML. Analyze audio, detect beats, and cache results for real-time timeline snapping.

- Repository: [Palmier/palmier-pro](https://github.com/palmier-io/palmier-pro)
- Tags: how-to-guide
- Published: 2026-07-20

---

**Palmier Pro analyzes audio assets to detect musical beats and downbeats using a Core ML pipeline, caching results in BeatStore to prevent redundant processing and enable real-time timeline snapping.**

Palmier Pro is a professional video editing application that provides precise audio beat detection to synchronize edits with musical rhythm. According to the palmier-io/palmier-pro source code, the system combines a singleton `BeatDetector` for ML inference with a `@MainActor`-bound `BeatStore` for intelligent caching. This architecture ensures that beat detection runs efficiently without blocking the UI while maintaining data consistency across the editor.

## The Beat Detection Pipeline

The beat detection workflow resides entirely in **[`BeatDetector.swift`](https://github.com/palmier-io/palmier-pro/blob/main/BeatDetector.swift)**, implemented as a singleton (`BeatDetector.shared`) that manages concurrent access via an `AsyncSemaphore` named `pipelineGate`, limiting execution to two simultaneous detections.

### Audio Decoding and Resampling

The process begins with `decodeAudio(from:)`, which uses `AVAssetReader` to extract a mono, 32-bit float PCM stream at **22.05 kHz**. The method returns raw samples as a flat `[Float]` array optimized for the Core ML model's expected input format.

### Chunked Inference with Overlapping Windows

The ML model processes audio in fixed-size chunks defined by `chunkFrames = 1500`. The `predictLogits(samples:model:)` method slides an overlapping window across the sample buffer, padding the start with six frames to ensure the first interior frame contains valid context. For each chunk, the Core ML model outputs per-frame logits representing the probability of **beat** and **downbeat** presence.

### Logit Aggregation and Peak Picking

For every interior frame, the system keeps the highest-confidence logit values from overlapping chunks using a `keep_first` strategy. This produces two aligned time-series (`beat` and `downbeat`) at **20 ms** resolution. The `pickPeaks(_:)` method converts these logits to probabilities via a sigmoid function, then extracts local maxima above a **0.5 threshold** to identify actual beat timestamps in seconds.

### BPM Estimation and Result Struct

After peak detection, `estimateBPM` calculates the median interval between successive beats to determine the approximate tempo. The final result wraps into a `BeatAnalysis` value type:

```swift
struct BeatAnalysis: Codable, Sendable, Equatable {
    let bpm: Double            // 0 when indeterminate
    let beats: [Double]        // seconds in source media
    let downbeats: [Double]
}

```

## BeatStore Caching Architecture

While `BeatDetector` handles the computational heavy lifting, **[`BeatStore.swift`](https://github.com/palmier-io/palmier-pro/blob/main/BeatStore.swift)** provides an in-memory cache and task coordination layer marked with `@MainActor`. This class prevents redundant analysis when multiple UI components request beat data for the same asset.

### Cache Structure and File Tagging

`BeatStore` maintains three primary dictionaries:

- **`analyses`**: Maps media-reference strings to their cached `BeatAnalysis` instances
- **`fileTags`**: Stores lightweight "size + modification time" fingerprints to detect source file changes
- **`tasks`** and **`hydrationTasks`**: Track in-flight detection or cache-hydration jobs to enable work sharing

The `fileTag` validation ensures that modifications to the underlying media trigger fresh analysis rather than returning stale cached data.

### Hydration vs. Fresh Detection

The store exposes distinct methods for different access patterns:

- **`analysis(for:)`**: Returns a cached `BeatAnalysis` synchronously if available
- **`hydrate(for:)`**: Asynchronously loads a previously saved analysis from `DiskCache` without invoking the ML detector
- **`detect(for:force:)`**: Starts a detection task (or returns an existing one) and stores the result. The `force` parameter bypasses file-tag guards to trigger re-analysis

### Concurrency and Task Deduplication

When `detect(for:)` receives multiple requests for the same asset before the first completes, the store returns the existing task handle rather than spawning duplicate work. The `invalidate(_:)` method clears cached data and cancels pending tasks for a specific media reference, while `reset()` empties the entire store—typically called when closing the editor or opening a new project.

## Timeline Integration and UI

The beat data flows into the user interface through **[`MediaVisualCache.swift`](https://github.com/palmier-io/palmier-pro/blob/main/MediaVisualCache.swift)**, which holds the `BeatStore` instance and exposes `analysis(for:)` to the rest of the editor.

### Triggering Detection from the UI

When users select **Detect Beats** from the timeline context menu (defined in **`TimelineView+BeatsMenu.swift`**), the editor calls `mediaVisualCache.beats.detect(for:asset, force:)`. This either returns a cached result or initiates the full ML pipeline via `BeatDetector.analysis(for:mediaRef:force:)`.

Upon completion, `BeatStore` executes the `onBeatsReady` closure to notify the timeline view, which triggers a repaint of beat ticks.

### Beat Snapping and Visual Rendering

The timeline visualization draws beat markers using **[`ClipRenderer.swift`](https://github.com/palmier-io/palmier-pro/blob/main/ClipRenderer.swift)**, specifically in `drawBeatTicks`, which reads the cached `BeatAnalysis`. For editing operations, **[`SnapEngine.swift`](https://github.com/palmier-io/palmier-pro/blob/main/SnapEngine.swift)** consumes beat timestamps through the `beatSnapFrames(for:)` extension method (defined on `EditorViewModel`), converting beat seconds into frame numbers based on the clip's FPS for precise beat-aligned snapping.

## Implementation Examples

**Starting a detection or reusing an existing task:**

```swift
let asset: MediaAsset = // the audio asset to analyze
let beatTask = editor.mediaVisualCache.beats.detect(for: asset)

Task {
    do {
        let analysis = try await beatTask.value
        print("BPM:", analysis.bpm)
        print("Beats at:", analysis.beats)
    } catch {
        print("Beat detection failed:", error)
    }
}

```

**Loading a previously saved analysis without running the detector:**

```swift
if let hydrateTask = editor.mediaVisualCache.beats.hydrate(for: asset) {
    // Hydration runs in the background; the store calls onBeatsReady when complete
}

```

**Getting snap frames for timeline alignment:**

```swift
let frames = editor.beatSnapFrames(for: clip)   // Array of Int frame numbers

```

## Summary

- **Audio Beat Detection** in Palmier Pro processes 22.05 kHz mono PCM through a chunked Core ML pipeline that outputs beat and downbeat timestamps with 20 ms precision.
- **[`BeatDetector.swift`](https://github.com/palmier-io/palmier-pro/blob/main/BeatDetector.swift)** handles the complete ML workflow—from decoding to peak picking—while limiting concurrency to two simultaneous analyses via `AsyncSemaphore`.
- **[`BeatStore.swift`](https://github.com/palmier-io/palmier-pro/blob/main/BeatStore.swift)** provides a `@MainActor`-bound cache that deduplicates requests, validates file integrity through modification tags, and distinguishes between hydrating cached results and running fresh detection.
- The system integrates with the timeline through **`TimelineView+BeatsMenu.swift`** for UI triggers, **[`ClipRenderer.swift`](https://github.com/palmier-io/palmier-pro/blob/main/ClipRenderer.swift)** for visual beat ticks, and **[`SnapEngine.swift`](https://github.com/palmier-io/palmier-pro/blob/main/SnapEngine.swift)** for frame-accurate beat snapping.

## Frequently Asked Questions

### How does Palmier Pro prevent redundant beat detection for the same audio file?

The `BeatStore` class tracks in-flight tasks using the `tasks` dictionary, ensuring that multiple simultaneous requests for the same asset receive the same `Task` handle rather than spawning duplicate ML inference jobs. Once complete, results reside in the `analyses` dictionary for synchronous access.

### What triggers a fresh beat analysis instead of using cached results?

The `detect(for:force:)` method checks the `fileTag` dictionary to compare the asset's current size and modification time against stored metadata. If the file has changed or the `force` parameter is `true`, the system bypasses the cache and runs the full detection pipeline.

### How does the beat detection handle concurrent processing without blocking the UI?

`BeatDetector` uses an `AsyncSemaphore` named `pipelineGate` to limit active detections to two concurrent operations, while `BeatStore` runs on `@MainActor` but delegates heavy work to background tasks. The `onBeatsReady` callback notifies the UI asynchronously when results become available.

### What audio format does the beat detection pipeline require?

The `decodeAudio(from:)` method in [`BeatDetector.swift`](https://github.com/palmier-io/palmier-pro/blob/main/BeatDetector.swift) forces audio to **mono 32-bit float PCM at 22.05 kHz** using `AVAssetReader`, regardless of the source format, ensuring compatibility with the Core ML model's expected input dimensions.