How to Fine-Tune OpenSuperWhisper Models: A Complete Developer Guide

OpenSuperWhisper supports fine-tuned Whisper models by loading compatible GGML .bin files from the local whisper-models directory or remote URLs, requiring no source code changes to switch between custom models.

OpenSuperWhisper, developed by Starmel, provides a macOS-native interface for OpenAI's Whisper with built-in infrastructure for custom fine-tuned models. Integrating your own fine-tuned OpenSuperWhisper models leverages the app's WhisperModelManager and Settings architecture to handle domain-specific speech recognition without modifying the core transcription engine.

Understanding the Model Management Architecture

OpenSuperWhisper delegates model lifecycle management to three key components that work together to handle fine-tuned models seamlessly.

The WhisperModelManager Component

The WhisperModelManager class (defined in OpenSuperWhisper/WhisperModelManager.swift) creates and maintains the whisper-models directory within the app's Application Support folder. On first launch, it copies the bundled tiny model to this directory. The manager exposes modelsDirectory as a computed property that returns the canonical path (typically ~/Library/Application Support/com.starmel.OpenSuperWhisper/whisper-models), and provides getAvailableModels() to scan this directory for valid .bin files.

Settings Integration and Persistence

The SettingsViewModel (located in OpenSuperWhisper/Settings.swift) maintains the UI state for model selection. When a user selects a fine-tuned model, the system writes the path to AppPreferences.shared.selectedWhisperModelPath, then triggers TranscriptionService.shared.reloadModel(with:) to hot-swap the active Whisper engine without restarting the application.

The SettingsDownloadableModel struct (lines 538–570 in Settings.swift) defines metadata for remote models, including name, url, filename, and optional preferredLanguage overrides.

Preparing Your Fine-Tuned Model

OpenSuperWhisper does not contain training code. You must create fine-tuned models externally using whisper.cpp training scripts, producing GGML-compatible .bin files.

1. Install whisper.cpp – Clone the repository and build the trainer using make or cmake.

2. Prepare your dataset – Collect audio files and transcriptions in Whisper's JSON format.

3. Run the training process:

./trainer -m models/ggml-base.en.bin -t /path/to/dataset -o fine_tuned_en.bin

4. Verify the output – Confirm the resulting file is a valid GGML .bin that matches the format expected by whisper.cpp.

The Hebrew fine-tune bundled in OpenSuperWhisper serves as a reference implementation, defined in SettingsDownloadableModels around lines 563–570 with a Hugging Face URL.

Adding Fine-Tuned Models to OpenSuperWhisper

You have two integration paths: registering the model for automatic download, or manually placing the file in the models directory.

Method 1: Adding a Downloadable Model Entry

Host your .bin file on a stable URL (Hugging Face is recommended), then register it in the source code:

1. Open OpenSuperWhisper/Settings.swift and locate the SettingsDownloadableModels struct.

2. Add a new entry to the availableModels array:

SettingsDownloadableModel(
    name: "My-Custom-Turbo-V3",
    isDownloaded: false,
    url: URL(string: "https://huggingface.co/username/repo/resolve/main/my-custom-model.bin?download=true")!,
    size: 1234,                       // Size in MB for progress tracking
    description: "Domain-specific fine-tune for medical terminology",
    filename: "my-custom-model.bin", // Optional: defaults to URL-derived name
    preferredLanguage: "en"          // Optional: forces language selection
)

3. Rebuild the application using ./run.sh build.

The new entry appears under Settings → Model → Download Models with a progress bar for the download.

Method 2: Manual File Installation

For local testing or private models, copy the file directly:

1. Locate the runtime models directory:

let folder = WhisperModelManager.shared.modelsDirectory.path
// Returns: ~/Library/Application Support/com.starmel.OpenSuperWhisper/whisper-models

2. Copy your my-custom-model.bin into this directory.

3. Open Settings → Model – the file appears automatically in the Available models list.

Selecting and Verifying Your Fine-Tuned Model

Once integrated, activation requires a single selection:

• In the Model tab, select your custom entry (or the manually placed file).
• If the model declares a preferredLanguage (like the Hebrew model), the UI automatically switches the Transcription Language to match.
• The app calls TranscriptionService.shared.reloadModel(with:) to load the new weights immediately.

Verification steps:

1. Record a short sample using the global shortcut (⌘ + Shift + R by default).
2. Check that the transcription reflects your domain-specific vocabulary.
3. Enable Debug Mode in Advanced → Debug Options to view detailed console logs if transcription behavior differs from expectations.

Summary

• Fine-tuned models must be GGML .bin files compatible with whisper.cpp, produced by external training pipelines.
• Two integration methods exist: Add a SettingsDownloadableModel entry for remote downloads, or copy files directly to WhisperModelManager.shared.modelsDirectory.
• Automatic engine reloading occurs via TranscriptionService.shared.reloadModel(with:) when you select a model in the Settings UI.
• Language preferences can be embedded in model metadata to auto-select transcription languages.

Frequently Asked Questions

What file format does OpenSuperWhisper require for fine-tuned models?

OpenSuperWhisper requires GGML-formatted .bin files that are compatible with the whisper.cpp inference engine. These files are typically generated using the whisper.cpp training scripts or converted from PyTorch checkpoints using the whisper.cpp conversion tools.

Can I use fine-tuned models without rebuilding the application?

Yes. While adding entries to SettingsDownloadableModels requires a rebuild, you can use fine-tuned models immediately by copying the .bin file to the whisper-models directory located at ~/Library/Application Support/com.starmel.OpenSuperWhisper/whisper-models. The app automatically detects these files on launch without requiring recompilation.

Where does OpenSuperWhisper store downloaded models?

Downloaded models are stored in the directory returned by WhisperModelManager.shared.modelsDirectory, which resolves to ~/Library/Application Support/com.starmel.OpenSuperWhisper/whisper-models on macOS systems. This directory is created automatically on first launch if it does not exist.

How do I troubleshoot transcription issues with my fine-tuned model?

Enable Debug Mode in the Advanced → Debug Options section of the Settings UI. This activates verbose logging to the console, allowing you to verify that TranscriptionService successfully loaded your model via reloadModel(with:) and to inspect any inference errors emitted by the underlying whisper.cpp engine.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}