How FluidVoice Handles Real-Time Transcription Updates: A Deep Dive into the Streaming Pipeline

FluidVoice streams audio from the microphone through a timer-driven ASRService, processes chunks via a provider-agnostic TranscriptionProvider protocol, and updates the SwiftUI interface incrementally while the user is still speaking.

FluidVoice, an open-source macOS application by altic-dev, delivers low-latency live transcription by combining a thread-safe audio buffer with asynchronous streaming providers. Understanding how the codebase handles real-time transcription updates requires examining the data flow from microphone capture to UI rendering. The architecture centers on ASRService orchestrating the pipeline while concrete providers handle speech-to-text conversion.

The Streaming Pipeline Architecture

The transcription pipeline operates as a continuous loop that extracts audio chunks and dispatches them for processing without blocking the main thread.

Audio Buffering and Chunk Extraction

At the heart of the system lies ThreadSafeAudioBuffer, a circular buffer that collects raw PCM samples from the microphone. In Sources/Fluid/Services/ASRService.swift, the startStreamingTranscription() method launches a timer-driven task that repeatedly calls processStreamingChunk() to extract the newest audio data.

// ASRService – start the periodic streaming loop
private func startStreamingTranscription() {
    self.streamingTask?.cancel()
    guard self.isAsrReady else { return }
    self.streamingTask = Task { await self.runStreamingLoop() }
}

Before processing, the service validates that sufficient samples have accumulated by checking minimumStreamingPreviewSamples. This prevents the provider from receiving incomplete audio frames that would yield poor recognition accuracy.

The TranscriptionProvider Protocol

The TranscriptionProvider protocol defined in Sources/Fluid/Services/TranscriptionProvider.swift abstracts the underlying speech recognition engines. It declares transcribeStreaming(_:) for low-latency partial results, defaulting to transcribe(_:) if a specific provider lacks streaming capabilities.

Concrete implementations such as ParakeetRealtimeProvider, FluidAudioProvider, and AppleSpeechProvider override this method to return incremental text as soon as words are recognized, enabling the "live preview" experience characteristic of FluidVoice's real-time transcription updates.

Chunk Processing and Backpressure Handling

To maintain responsiveness, ASRService implements flow control mechanisms that prevent pipeline saturation. The service tracks an isProcessingChunk Boolean flag to ensure only one transcription request operates at a time.

// Called repeatedly – extracts a chunk and asks the provider for a partial result
@MainActor
private func processStreamingChunk() async {
    guard self.isRunning, !self.isProcessingChunk else { return }
    let chunk = self.audioBuffer.getPrefix(self.audioBuffer.count)
    self.isProcessingChunk = true
    defer { self.isProcessingChunk = false }

    let result = try await transcriptionExecutor.run {
        try await transcriptionProvider.transcribeStreaming(chunk)
    }
    // Forward partial result to the UI state
    NotchContentState.shared.updateTranscription(result.text)
}

If the previous chunk remains in flight when the timer fires, the service drops or skips frames rather than queueing redundant requests. This backpressure handling ensures that real-time transcription updates remain synchronized with the actual audio timeline, preventing the UI from displaying stale results.

From Provider to UI: Result Propagation

Once the TranscriptionProvider returns partial text, the result flows through a structured concurrency pipeline. The transcriptionExecutor—a Task-based wrapper executing on a background executor—handles the asynchronous work. Upon success, the singleton NotchContentState receives the update via updateTranscription(_:) in Sources/Fluid/Views/NotchContentViews.swift.

The state object applies a character limit using tailCharacters(in:maxCharacters:) to bound memory usage, then publishes changes through the @Published property transcriptionText. SwiftUI views including NotchExpandedView and BottomOverlayView automatically refresh when this property changes, displaying the latest words within milliseconds of recognition.

// NotchContentState – keep a bounded preview and publish it for SwiftUI
func updateTranscription(_ text: String) {
    let bounded = Self.tailCharacters(in: text,
                                      maxCharacters: Self.maxStoredTranscriptionCharacters)
    guard bounded != self.transcriptionText else { return }
    self.transcriptionText = bounded
    self.recomputeTranscriptionLines()
}

Key Implementation Files

The following source files implement the real-time transcription pipeline in FluidVoice:

• Sources/Fluid/Services/ASRService.swift – Orchestrates the streaming loop, buffer management, and chunk validation
• Sources/Fluid/Services/TranscriptionProvider.swift – Defines the protocol abstraction for all ASR backends
• Sources/Fluid/Views/NotchContentViews.swift – Houses NotchContentState for UI state management
• Sources/Fluid/Views/BottomOverlayView.swift – Renders the live preview overlay using the published transcription state
• Sources/Fluid/Services/ParakeetRealtimeProvider.swift – Concrete implementation providing low-latency streaming transcription

Summary

• Audio Capture: ASRService maintains a ThreadSafeAudioBuffer that accumulates microphone samples for processing.
• Streaming Protocol: The TranscriptionProvider protocol enables pluggable speech recognition via transcribeStreaming(_:), with concrete providers like ParakeetRealtimeProvider handling specific engine implementations.
• Flow Control: The service enforces minimumStreamingPreviewSamples and uses isProcessingChunk to prevent backpressure and dropped frames.
• UI Synchronization: NotchContentState publishes bounded transcription text via @Published properties, driving SwiftUI views such as BottomOverlayView with minimal latency.

Frequently Asked Questions

How does FluidVoice prevent audio buffer overflow during transcription?

FluidVoice prevents overflow through the isProcessingChunk flag in ASRService, which ensures only one transcription request processes at a time. If the provider is still processing a previous chunk when the timer fires, the service skips the current frame rather than queueing additional requests. Additionally, the minimumStreamingPreviewSamples check ensures chunks meet a size threshold before dispatching, reducing unnecessary provider calls.

What is the TranscriptionProvider protocol in FluidVoice?

The TranscriptionProvider protocol is the abstraction layer defined in Sources/Fluid/Services/TranscriptionProvider.swift that decouples the audio streaming logic from specific speech recognition engines. It requires conforming types to implement transcribeStreaming(_:) for partial results, with a default implementation falling back to transcribe(_:). This design allows FluidVoice to switch between providers like AppleSpeechProvider, FluidAudioProvider, and ParakeetRealtimeProvider without modifying the core streaming pipeline.

How does FluidVoice update the UI with partial transcription results?

Partial results propagate from the provider through ASRService to the NotchContentState singleton, which updates its @Published transcriptionText property. The updateTranscription(_:) method trims the incoming string to maxStoredTranscriptionCharacters to manage memory, then calls recomputeTranscriptionLines(). SwiftUI views such as NotchExpandedView and BottomOverlayView observe these published changes and refresh automatically, displaying words as they are recognized.

Which concrete providers implement real-time transcription in FluidVoice?

FluidVoice includes several concrete implementations of TranscriptionProvider that support streaming: ParakeetRealtimeProvider for optimized on-device recognition, FluidAudioProvider for the project's native engine, and AppleSpeechProvider for macOS system speech recognition. Each implements transcribeStreaming(_:) to return incremental text chunks, enabling the application to display words while the user continues speaking rather than waiting for final utterances.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}