# How the Whisper Engine is Integrated in OpenSuperWhisper: Complete Technical Guide

> Discover how the Whisper engine integrates into OpenSuperWhisper using a two-layer architecture. This technical guide details Swift class orchestration and C API interfacing.

- Repository: [Starmel/OpenSuperWhisper](https://github.com/Starmel/OpenSuperWhisper)
- Tags: technical-guide
- Published: 2026-07-07

---

**The Whisper engine in OpenSuperWhisper is implemented through a two-layer architecture: a high-level Swift class `WhisperEngine` that conforms to the `TranscriptionEngine` protocol and orchestrates the transcription pipeline, and a low-level `MyWhisperContext` wrapper that interfaces with the native Whisper C API.**

The OpenSuperWhisper repository provides a native macOS transcription application that leverages OpenAI's Whisper models for offline speech-to-text conversion. Understanding how the Whisper engine is integrated in OpenSuperWhisper reveals a clean separation between the Swift-based application layer and the underlying C++ inference engine, bridged through careful audio processing and callback management.

## Architecture Overview

The integration follows a protocol-based design pattern. The `WhisperEngine` class defined in [`OpenSuperWhisper/Engines/WhisperEngine.swift`](https://github.com/Starmel/OpenSuperWhisper/blob/main/OpenSuperWhisper/Engines/WhisperEngine.swift) implements the `TranscriptionEngine` protocol, providing a standardized interface for transcription operations. This engine interacts with the native Whisper library through `MyWhisperContext`, a thin Swift wrapper located in [`OpenSuperWhisper/Whis/Whis.swift`](https://github.com/Starmel/OpenSuperWhisper/blob/main/OpenSuperWhisper/Whis/Whis.swift) that exposes C API functions including model initialization, inference execution, and segment retrieval.

## Core Implementation Components

### WhisperEngine.swift - The High-Level Orchestrator

Located at [`OpenSuperWhisper/Engines/WhisperEngine.swift`](https://github.com/Starmel/OpenSuperWhisper/blob/main/OpenSuperWhisper/Engines/WhisperEngine.swift), this file contains the primary Swift interface. The `WhisperEngine` class manages the complete transcription lifecycle through six distinct stages:

1. **Model Loading** – The `initialize()` method reads the selected model file and instantiates a `MyWhisperContext` (lines 66-73).
2. **Audio Conversion** – `convertAudioToPCM(_:)` transforms input audio into 16 kHz mono Float-32 PCM buffers required by Whisper (lines 39-84).
3. **Parameter Preparation** – The engine populates a `WhisperFullParams` struct using values from the UI `Settings` object, including language detection, beam search configuration, and temperature sampling (lines 16-28).
4. **Progress Callbacks** – Custom C-compatible callbacks forward Whisper's internal progress (0-100%) to Swift via a `ProgressContext` object, enabling real-time UI updates (lines 29-55).
5. **Inference Execution** – The `context.full(samples:params:)` method executes the actual Whisper inference on the prepared PCM samples (lines 71-74).
6. **Result Assembly** – Post-transcription processing iterates over segments, optionally injects timestamps, cleans formatting markers, and applies Asian-language autocorrection (lines 78-107).

### Whis.swift - The C API Bridge

The [`OpenSuperWhisper/Whis/Whis.swift`](https://github.com/Starmel/OpenSuperWhisper/blob/main/OpenSuperWhisper/Whis/Whis.swift) file implements `MyWhisperContext`, which wraps the native Whisper C library. This wrapper exposes critical methods including `initFromFile` for model loading, `full` for running inference, and segment getters such as `fullNSegments` used by the higher-level engine (lines 18-45). This abstraction layer isolates the Swift codebase from direct C struct management and memory handling.

## Transcription Pipeline Implementation

### Audio Preprocessing

Before inference, the engine must standardize input formats. The `convertAudioToPCM(_:)` method handles format conversion, resampling arbitrary audio inputs to the 16 kHz mono Float-32 PCM specification required by Whisper's encoder.

### Configuration Management

Transcription parameters flow from the user interface through the `Settings` object into the `WhisperFullParams` structure. This includes decoding strategies like beam search width, temperature sampling for diversity control, and language auto-detection settings.

### Real-Time Progress Handling

The integration implements bidirectional communication between Swift and C through progress callbacks. The `ProgressContext` object bridges Whisper's internal progress reporting to Swift closures, enabling the UI to display transcription percentages while maintaining the ability to cancel operations mid-stream.

## Usage Example

To utilize the Whisper engine in OpenSuperWhisper, applications interact with the `TranscriptionService` which manages engine lifecycle:

```swift
// Initialize the engine (typically handled by TranscriptionService)
await WhisperEngine().initialize()

// Prepare transcription parameters
let audioURL = URL(fileURLWithPath: "/path/to/audio.wav")
let settings = Settings() // UI-provided configuration

// Execute transcription with progress monitoring
let engine = WhisperEngine()
engine.onProgressUpdate = { progress in
    print("Transcription progress: \(progress * 100)%")
}

let transcript = try await engine.transcribeAudio(url: audioURL, settings: settings)

```

## Summary

- **Two-Layer Architecture**: OpenSuperWhisper integrates Whisper through `WhisperEngine` (Swift orchestration) and `MyWhisperContext` (C API binding).
- **Protocol-Based Design**: The `TranscriptionEngine` protocol enables interchangeable transcription backends.
- **Audio Standardization**: All inputs convert to 16 kHz mono Float-32 PCM before processing.
- **Callback Integration**: C-compatible progress callbacks enable real-time UI updates during inference.
- **Result Processing**: Post-processing includes timestamp generation, marker cleanup, and Asian-language autocorrection.

## Frequently Asked Questions

### Where is the Whisper engine code located in OpenSuperWhisper?

The core implementation resides in two main files: [`OpenSuperWhisper/Engines/WhisperEngine.swift`](https://github.com/Starmel/OpenSuperWhisper/blob/main/OpenSuperWhisper/Engines/WhisperEngine.swift) contains the high-level Swift orchestration logic, while [`OpenSuperWhisper/Whis/Whis.swift`](https://github.com/Starmel/OpenSuperWhisper/blob/main/OpenSuperWhisper/Whis/Whis.swift) houses the `MyWhisperContext` wrapper that binds to the native Whisper C library.

### How does OpenSuperWhisper handle audio format compatibility?

The `WhisperEngine` class automatically converts all input audio to 16 kHz mono Float-32 PCM format through the `convertAudioToPCM(_:)` method, ensuring compatibility with Whisper's encoder requirements regardless of the source file format.

### Can the transcription process be monitored or cancelled in real-time?

Yes. The implementation uses C-compatible callbacks that forward Whisper's internal progress (0-100%) to a `ProgressContext` object, which updates the UI. This mechanism also supports cancellation tokens that can abort inference mid-operation.

### What parameters can be configured for Whisper transcription?

The engine accepts configuration through the `Settings` object, which populates the `WhisperFullParams` struct with options including language selection, beam search settings, temperature sampling, and timestamp generation preferences.