What Kind of Data Does Palmier Pro Process? A Deep Dive into the AI-Native Video Editor

Palmier Pro processes multimedia files (video, audio, images, and Lottie JSON), text overlays, speech transcriptions, and AI-generated assets, organizing them into a timeline-driven project model persisted as JSON.

Palmier Pro is an AI-native macOS video editor that ingests diverse media types and auxiliary data to create structured, editable projects. According to the palmier-io/palmier-pro source code, the application handles everything from raw video files to on-device speech transcripts and AI-generated clips. This analysis explores the specific data types processed by the editor and how they flow through the Swift/AVFoundation architecture.

Core Media Types: Video, Audio, and Images

Palmier Pro handles video, audio, and static images as foundational media assets. Each type follows a specific ingestion pipeline defined in Sources/PalmierPro/Models/MediaAsset.swift.

Video data loads as AVURLAsset and undergoes extraction for thumbnails, waveform generation, and composition. In Sources/PalmierPro/Preview/VideoEngine.swift, the rebuild() method constructs an AVComposition from video clips, while MediaAsset stores critical metadata including sourceWidth, sourceHeight, sourceFPS, and hasAudio.

Audio is treated either as a standalone track or as the audio component of a video asset. The system generates waveforms using DSWaveformImage, with caching handled in MediaVisualCache.swift.

Images import as static visual assets with thumbnails generated via ImageEncoder. These become visual tracks (for example, picture-in-picture overlays) and are classified by type == .image in the data model.

Text Overlays and Animation Data

Beyond traditional media, Palmier Pro processes text and Lottie animations as editable layers.

Text clips overlay video through TextLayerController.swift, which manages TextLayerController for drawing and animating text synchronized with the timeline.

Lottie (JSON animation) files are parsed by LottieVideoGenerator.swift into temporary .mov files. This conversion allows Lottie animations to play back like standard video assets during preview and export operations.

AI-Generated Content and Speech Data

The editor processes two specialized data categories: transcribed speech and AI-generated media.

Transcribed Speech is generated via on-device speech-to-text using Apple's Speech framework. The Transcription.swift file extracts audio from video files and runs SpeechTranscriber to return a TranscriptionResult containing full text, per-word timestamps, and segmented utterances.

Generated Media represents AI-created clips (such as video-to-audio edits or generative video). These are stored as GenerationInput objects in Sources/PalmierPro/Generation/Edit/GenerationInput.swift and tracked with a GenerationStatus flag within MediaAsset.swift.

Project Structure and Timeline Data

Palmier Pro organizes all processed data into structured project files.

Project Manifest: A JSON manifest (MediaManifest.json) records every asset, its metadata, folder structure, and cached remote URLs. The VideoProject.swift file handles reading and writing this manifest as an NSDocument subclass.

Timeline and Tracks: A declarative Timeline model (defined in Timeline.swift) describes tracks, clips, framerate, and composition size. The editor manipulates this data structure while the preview engine renders it to an AVComposition via VideoEngine.rebuild().

Export Packages: When exporting, ExportService.swift assembles a package containing the timeline JSON, media manifest, generated thumbnails, and optional chat-session data from AgentService.swift.

Practical Code Examples

Loading a Media Asset

let asset = MediaAsset(
    id: UUID().uuidString,
    url: URL(fileURLWithPath: "/path/to/file.mov"),
    type: .video,
    name: "My Clip",
    duration: 0
)
editorViewModel.mediaAssets.append(asset)
await asset.loadMetadata()               // pulls width/height/audio info
editorViewModel.mediaVisualCache.generateWaveform(for: asset)

This pattern from MediaAsset.swift initializes assets with metadata extraction via loadMetadata().

Generating Video Thumbnails

let generator = AVAssetImageGenerator(asset: AVURLAsset(url: asset.url))
generator.maximumSize = CGSize(width: 320, height: 180)
generator.appliesPreferredTrackTransform = true
let time = CMTime(value: CMTimeValue(clip.trimStartFrame),
                  timescale: CMTimeScale(editorViewModel.timeline.fps))
let cgImage = try await generator.image(at: time).image
let thumbnail = NSImage(cgImage: cgImage,
                        size: NSSize(width: cgImage.width,
                                     height: cgImage.height))
asset.thumbnail = thumbnail

This implementation appears in VideoEngine.swift within the captureThumbnail() method.

Running On-Device Transcription

let result = try await Transcription.transcribeVideoAudio(
    videoURL: URL(fileURLWithPath: "/path/to/video.mov"),
    censorProfanity: true,
    preferredLocale: Locale(identifier: "en-US")
)
// `result.words` contains per‑word timestamps; `result.text` is the full transcript.

The Transcription.swift file manages the full pipeline from audio extraction to speech recognition.

Exporting a Project Package

try ExportService.export(project: editorViewModel,
                         to: URL(fileURLWithPath: "/tmp/MyProject.palmier"))

ExportService.swift assembles ProjectPackageSnapshot and writes the JSON timeline, manifest, and media folder.

Summary

• Palmier Pro processes video, audio, images, Lottie JSON, text overlays, speech transcriptions, and AI-generated media through a unified pipeline.
• The MediaAsset.swift model centralizes metadata handling for all imported and generated assets.
• Speech transcription runs on-device via Transcription.swift using Apple's Speech framework.
• Lottie animations convert to temporary video files via LottieVideoGenerator.swift for seamless timeline integration.
• Project data persists as JSON through VideoProject.swift, including the media manifest and timeline structure.
• The export system packages all data types—including optional AI agent chat logs from AgentService.swift—into portable project files.

Frequently Asked Questions

Does Palmier Pro process audio separately from video?

Yes. While audio can be part of a video asset (tracked via the hasAudio flag in MediaAsset.swift), the system also treats audio as standalone tracks. The DSWaveformImage library generates visual waveforms for audio assets, and VideoEngine.swift manages audio tracks separately within the AVComposition during playback and export.

How does Palmier Pro handle AI-generated content?

AI-generated clips are stored as GenerationInput objects defined in GenerationInput.swift and tracked with a GenerationStatus enumeration within MediaAsset.swift. This architecture supports video-to-audio edits and other generative media, allowing the editor to distinguish between imported files and AI-created assets while maintaining the same timeline integration capabilities.

What is the project manifest in Palmier Pro?

The project manifest is a JSON file (MediaManifest.json) that records every asset, its metadata, folder structure, and cached remote URLs. VideoProject.swift manages this manifest as part of its NSDocument implementation, ensuring that all processed data—from video clips to agent chat sessions—can be saved, reopened, and shared across sessions.

How does speech transcription work in Palmier Pro?

The Transcription.swift file extracts audio from video files and runs the on-device SpeechTranscriber using Apple's Speech framework. The process returns a TranscriptionResult containing the full transcript text and per-word timestamps, enabling features like searchable dialogue and precise text-based editing without requiring cloud processing.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}