# How to Extend or Modify the Functionality of the Whisper Engine in OpenSuperWhisper

> Learn how to extend or modify the Whisper engine in OpenSuperWhisper by adjusting Swift wrappers, parameters, or implementing custom processing pipelines for enhanced functionality.

- Repository: [Starmel/OpenSuperWhisper](https://github.com/Starmel/OpenSuperWhisper)
- Tags: how-to-guide
- Published: 2026-07-05

---

**Extend the Whisper engine by modifying the Swift wrapper in [`Whis.swift`](https://github.com/Starmel/OpenSuperWhisper/blob/main/Whis.swift), adjusting parameters in [`Settings.swift`](https://github.com/Starmel/OpenSuperWhisper/blob/main/Settings.swift), or implementing the `TranscriptionEngine` protocol to create custom processing pipelines.**

OpenSuperWhisper provides a modular macOS transcription framework built atop the [`whisper.cpp`](https://github.com/Starmel/OpenSuperWhisper/blob/main/whisper.cpp) C library. To extend or modify the functionality of the Whisper engine, developers work through three distinct architectural layers: the low-level C wrapper, the high-level engine implementation, and the configuration interface.

## Understanding the Three-Tier Architecture

The codebase separates concerns into a **C-wrapper**, an **engine implementation**, and a **configuration layer**.

### The C Wrapper Layer (MyWhisperContext)

At the foundation, [[`Whis.swift`](https://github.com/Starmel/OpenSuperWhisper/blob/main/Whis.swift)](https://github.com/Starmel/OpenSuperWhisper/blob/master/OpenSuperWhisper/Whis/Whis.swift) contains `MyWhisperContext`, which provides a thin Swift façade over the native [`whisper.cpp`](https://github.com/Starmel/OpenSuperWhisper/blob/main/whisper.cpp) API. This class manages the `whisper_context` pointer, handles mel-spectrogram generation through `pcmToMel`, and performs token encoding and decoding. It maintains both a primary `ctx` and an optional `state` for stateful inference scenarios.

### The Engine Implementation (WhisperEngine)

The [[`WhisperEngine.swift`](https://github.com/Starmel/OpenSuperWhisper/blob/main/WhisperEngine.swift)](https://github.com/Starmel/OpenSuperWhisper/blob/master/OpenSuperWhisper/Engines/WhisperEngine.swift) file implements the `TranscriptionEngine` protocol, orchestrating the complete transcription workflow. The engine initializes the model via `MyWhisperContext.initFromFile`, converts audio to PCM format, maps `Settings` values to `WhisperFullParams`, and bridges C-side progress callbacks through `ProgressContext`.

### The Configuration Layer (Settings)

User preferences are centralized in [[`Settings.swift`](https://github.com/Starmel/OpenSuperWhisper/blob/main/Settings.swift)](https://github.com/Starmel/OpenSuperWhisper/blob/master/OpenSuperWhisper/Settings.swift), which defines all configurable parameters—from language selection to beam search strategies. These values flow directly into the `WhisperFullParams` structure during the transcription initialization phase.

## Extension Strategies

You can modify the engine through several pathways depending on your use case.

### Adding Custom Whisper Parameters

To expose additional [`whisper.cpp`](https://github.com/Starmel/OpenSuperWhisper/blob/main/whisper.cpp) parameters (such as `max_len`), extend the configuration and mapping pipeline:

1. **Extend `Settings`** by adding a new property with a default value:

```swift
// Settings.swift
var maxTokenLength: Int = 225

```

2. **Map the property** in `WhisperEngine.transcribeAudio` where `params` are constructed:

```swift
// WhisperEngine.swift
params.max_len = Int32(settings.maxTokenLength)

```

3. **Update the UI** in `SettingsView` to expose a control bound to this property.

### Modifying Audio Preprocessing

The `convertAudioToPCM` method in [[`WhisperEngine.swift`](https://github.com/Starmel/OpenSuperWhisper/blob/main/WhisperEngine.swift)](https://github.com/Starmel/OpenSuperWhisper/blob/master/OpenSuperWhisper/Engines/WhisperEngine.swift) handles audio format conversion. Insert custom DSP logic before the PCM buffer reaches the Whisper context:

```swift
private func applyNoiseReduction(to samples: [Float]) -> [Float] {
    let alpha: Float = 0.9
    var previous: Float = 0
    var out = [Float]()
    out.reserveCapacity(samples.count)
    for s in samples {
        let filtered = s - alpha * previous
        out.append(filtered)
        previous = s
    }
    return out
}

// Usage inside convertAudioToPCM
let cleanSamples = applyNoiseReduction(to: samples)

```

### Creating Custom Transcription Engines

For radically different behavior (such as streaming or alternative backends), implement the [`TranscriptionEngine`](https://github.com/Starmel/OpenSuperWhisper/blob/master/OpenSuperWhisper/Engines/TranscriptionEngine.swift) protocol:

```swift
// StreamingEngine.swift
class StreamingEngine: TranscriptionEngine {
    var engineName: String { "Streaming Whisper" }
    private var context: MyWhisperContext?
    var onProgressUpdate: ((Float) -> Void)?

    func initialize() async throws {
        let path = AppPreferences.shared.selectedWhisperModelPath!
        let params = WhisperContextParams()
        guard let ctx = MyWhisperContext.initFromFile(path: path, params: params) else {
            throw TranscriptionError.contextInitializationFailed
        }
        context = ctx
    }

    func processChunk(_ pcm: [Float]) throws -> String {
        guard let ctx = context else { throw TranscriptionError.contextInitializationFailed }
        // Implement encode/decode logic for streaming buffers
        return ""
    }

    func cancelTranscription() { /* Set abort flag */ }
    func getSupportedLanguages() -> [String] { LanguageUtil.availableLanguages }
}

```

Register the new engine in the UI layer where `WhisperEngine` is currently instantiated.

### Implementing Post-Processing Hooks

To modify transcription output (e.g., adding punctuation restoration or text normalization), hook into the result assembly phase in `WhisperEngine.transcribeAudio`. After the text segments are collected but before returning, apply your transformations:

```swift
let rawText = // ... assembled from segments
let processedText = applyCustomPostProcessing(rawText)
return processedText

```

## Summary

- **Architecture**: The engine consists of `MyWhisperContext` (C wrapper), `WhisperEngine` (orchestration), and `Settings` (configuration).
- **Parameter Extension**: Add fields to `Settings` and map them to `WhisperFullParams` in the engine initialization.
- **Audio Pipeline**: Override `convertAudioToPCM` to insert custom DSP filters before inference.
- **Custom Engines**: Conform to `TranscriptionEngine` to implement alternative transcription strategies or streaming support.
- **Model Management**: Extend [[`WhisperModelManager.swift`](https://github.com/Starmel/OpenSuperWhisper/blob/main/WhisperModelManager.swift)](https://github.com/Starmel/OpenSuperWhisper/blob/master/OpenSuperWhisper/WhisperModelManager.swift) to support new model formats or download sources.

## Frequently Asked Questions

### How do I add a new parameter to the Whisper engine?

Add the property to the `Settings` struct in [[`Settings.swift`](https://github.com/Starmel/OpenSuperWhisper/blob/main/Settings.swift)](https://github.com/Starmel/OpenSuperWhisper/blob/master/OpenSuperWhisper/Settings.swift), then map it to the corresponding field in `WhisperFullParams` within [[`WhisperEngine.swift`](https://github.com/Starmel/OpenSuperWhisper/blob/main/WhisperEngine.swift)](https://github.com/Starmel/OpenSuperWhisper/blob/master/OpenSuperWhisper/Engines/WhisperEngine.swift) during the transcription setup. Expose the control in the settings UI to make it user-configurable.

### Can I replace the audio preprocessing pipeline?

Yes. Modify the `convertAudioToPCM` method in [[`WhisperEngine.swift`](https://github.com/Starmel/OpenSuperWhisper/blob/main/WhisperEngine.swift)](https://github.com/Starmel/OpenSuperWhisper/blob/master/OpenSuperWhisper/Engines/WhisperEngine.swift) to insert custom DSP steps—such as noise reduction, normalization, or format conversion—before the PCM data is passed to `MyWhisperContext.full`.

### How do I create a completely new transcription engine?

Create a new class that conforms to the [`TranscriptionEngine`](https://github.com/Starmel/OpenSuperWhisper/blob/master/OpenSuperWhisper/Engines/TranscriptionEngine.swift) protocol, implementing `initialize()`, `transcribeAudio()`, `cancelTranscription()`, and `getSupportedLanguages()`. You can reuse `MyWhisperContext` for low-level operations or implement a custom backend entirely.

### Where are language models managed?

Model discovery, downloading, and validation are handled in [[`WhisperModelManager.swift`](https://github.com/Starmel/OpenSuperWhisper/blob/main/WhisperModelManager.swift)](https://github.com/Starmel/OpenSuperWhisper/blob/master/OpenSuperWhisper/WhisperModelManager.swift). To support new model formats, update the download URL validation logic and ensure the model file compatibility checks align with your new format requirements.