FluidAudio vs FluidAudioTTS: Understanding the Difference Between the Core SDK and Advanced TTS Add-On

FluidAudio is the GPL-free core SDK providing on-device ASR, diarization, VAD, and PocketTTS, while FluidAudioTTS is an optional add-on that adds the Kokoro TTS engine with SSML support at the cost of a GPL-3.0 dependency.

The fluidinference/fluidaudio repository distributes two distinct Swift package products designed for different licensing and functionality requirements. Both products share the same underlying speech AI infrastructure—including Parakeet ASR models and Silero VAD—but diverge significantly in their text-to-speech capabilities and dependency chains.

Core Architecture and Licensing Differences

The primary distinction between FluidAudio and FluidAudioTTS lies in their licensing constraints and TTS backend implementations.

FluidAudio: The GPL-Free Core SDK

The FluidAudio product is the foundation of the SDK, exported as .product(name: "FluidAudio", …) in the Swift package manifest. According to the repository's README.md (lines 99-105), this product bundles:

• Parakeet ASR models (TDT v3/v2) for automatic speech recognition
• Speaker diarization pipelines (offline and Sortformer-based)
• Silero VAD for voice activity detection
• PocketTTS – a flow-matching language model TTS that converts text directly to audio tokens without phoneme conversion

Crucially, FluidAudio contains no GPL-licensed dependencies, making it suitable for closed-source commercial applications. The entire product is licensed under Apache 2.0.

FluidAudioTTS: The Advanced TTS Add-On

The FluidAudioTTS product is an optional extension that layers advanced TTS capabilities atop the core SDK. As documented in README.md (lines 106-110), importing this product adds:

• Kokoro – a larger Core ML TTS model supporting phoneme-based synthesis, SSML markup, and IPA phoneme control
• eSpeak NG – a GPL-3.0-licensed C library required by Kokoro for grapheme-to-phoneme conversion of out-of-vocabulary words

Because FluidAudioTTS links against eSpeak NG (GPL-3.0), it is distributed as a separate Swift package product to prevent GPL contamination of the core SDK. Developers should only import FluidAudioTTS when requiring Kokoro's advanced features like SSML support, custom lexicons, or multi-voice synthesis.

Technical Capabilities Comparison

Capability	FluidAudio (Core)	FluidAudioTTS (Add-On)
ASR Engine	Parakeet TDT v3/v2 Core ML models	Inherited from core
Speaker Diarization	Offline + Sortformer pipelines	Inherited from core
VAD	Silero VAD	Inherited from core
TTS Backend	PocketTTS – flow-matching LM, text → tokens → audio (no phoneme step)	Kokoro – text → (optional eSpeak NG phoneme conversion) → IPA → token mapping → audio
SSML Support	No	Yes (full SSML + IPA phoneme markup)
License	Apache 2.0 (GPL-free)	Apache 2.0 core + GPL-3.0 (eSpeak NG)
Swift Package Product	.product(name: "FluidAudio", …)	.product(name: "FluidAudioTTS", …)

The architectural divergence is evident in the source file organization. PocketTTS model identifiers are defined in Sources/FluidAudio/ModelNames.swift (lines 311-328), while Kokoro-specific implementations reside in Sources/FluidAudioEspeak/TextToSpeech/KokoroTtsManager.swift, which orchestrates the lexicon cache, eSpeak NG phoneme conversion, and multi-stage synthesis pipeline.

Implementation Examples

Using FluidAudio Core (PocketTTS)

The following example demonstrates the GPL-free core SDK with PocketTTS for simple text-to-speech synthesis:

import FluidAudio

Task {
    // Initialize ASR components
    let asrModels = try await AsrModels.downloadAndLoad(version: .v3)
    let asr = AsrManager(config: .default)
    try await asr.initialize(models: asrModels)
    
    // Transcribe audio
    let audio = try AudioConverter().resampleAudioFile(path: "meeting.wav")
    let transcript = try await asr.transcribe(audio)
    print("Transcript: \(transcript.text)")
    
    // Synthesize speech with PocketTTS (GPL-free)
    let tts = PocketTtsManager()
    try await tts.initialize()
    let wav = try await tts.synthesize(text: "Hello from FluidAudio.")
    try wav.write(to: URL(fileURLWithPath: "hello.wav"))
}

All classes (AsrManager, PocketTtsManager) belong to the FluidAudio product, implemented in Sources/FluidAudio/TTS/PocketTTS/PocketTtsManager.swift.

Using FluidAudioTTS (Kokoro with SSML)

For applications requiring advanced voice control, import the FluidAudioTTS product to access Kokoro with SSML and phoneme markup:

import FluidAudioTTS

Task {
    // Initialize Kokoro manager (downloads models and lexicons)
    let kokoro = KokoroTtsManager()
    try await kokoro.initialize()
    
    // SSML with IPA phoneme markup and voice selection
    let ssml = """
    <voice name="alba">
        <phoneme alphabet="ipa" ph="kəˈkɔɹo">Kokoro</phoneme> is speaking.
    </voice>
    """
    
    // Synthesize with full SSML support
    let wav = try await kokoro.synthesize(ssml: ssml)
    try wav.write(to: URL(fileURLWithPath: "kokoro.wav"))
}

KokoroTtsManager resides in Sources/FluidAudioEspeak/TextToSpeech/KokoroTtsManager.swift and is only available when linking against the FluidAudioTTS product, which pulls in the GPL-3.0-licensed eSpeak NG dependency via Sources/FluidAudioEspeak/TextToSpeech/TtsBackend.swift.

Summary

• FluidAudio is the core, GPL-free Swift package product providing ASR (Parakeet), diarization, VAD (Silero), and PocketTTS for on-device speech AI without licensing restrictions.
• FluidAudioTTS is an optional add-on that introduces the Kokoro TTS engine with SSML support and phoneme conversion, but requires linking against GPL-3.0-licensed eSpeak NG.
• Choose FluidAudio for closed-source commercial apps requiring transcription and simple TTS; choose FluidAudioTTS only when you need advanced voice control features like SSML markup, custom lexicons, or multi-voice synthesis and can comply with GPL-3.0 requirements.

Frequently Asked Questions

What is the licensing difference between FluidAudio and FluidAudioTTS?

FluidAudio is distributed under the Apache 2.0 license and contains no GPL dependencies, making it safe for closed-source commercial applications. FluidAudioTTS includes eSpeak NG, which is licensed under GPL-3.0, meaning any app linking against FluidAudioTTS must comply with GPL-3.0 requirements, including source code disclosure for distributed binaries.

Can I use FluidAudio without FluidAudioTTS for text-to-speech?

Yes. FluidAudio includes PocketTTS, a flow-matching language model TTS that converts text directly to audio without phoneme conversion. While it lacks SSML support and advanced voice control, PocketTTS provides high-quality synthesis sufficient for most applications without the GPL licensing burden of FluidAudioTTS.

When should I choose FluidAudioTTS over the core FluidAudio package?

Choose FluidAudioTTS when your application requires Kokoro's advanced features: SSML markup for fine-grained prosody control, IPA phoneme customization, multi-voice switching, or custom lexicons for domain-specific vocabulary. If you only need basic TTS alongside ASR and diarization, the core FluidAudio package provides a simpler, GPL-free alternative.

Does FluidAudioTTS provide better ASR or diarization than FluidAudio?

No. Both products share identical ASR (Parakeet), diarization (offline and Sortformer), and VAD (Silero) capabilities. The only difference is the TTS backend: FluidAudio provides PocketTTS, while FluidAudioTTS adds Kokoro with eSpeak NG integration. All other speech AI functionality remains consistent between the two Swift package products.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}