# NemotronProvider in FluidVoice: Features, Implementation, and Usage Guide

> Explore NemotronProvider in FluidVoice. Get high-accuracy offline and ultra-low-latency streaming speech recognition exclusively on Apple Silicon for NVIDIA Nemotron 3.5 models.

- Repository: [ALTIC/FluidVoice](https://github.com/altic-dev/FluidVoice)
- Tags: how-to-guide
- Published: 2026-07-07

---

**The NemotronProvider is FluidVoice's native transcription engine for NVIDIA Nemotron 3.5 models, delivering high-accuracy offline and ultra-low-latency streaming speech recognition exclusively on Apple Silicon devices.**

The `NemotronProvider` class in the [altic-dev/FluidVoice](https://github.com/altic-dev/FluidVoice) repository implements the `TranscriptionProvider` protocol from [`Sources/Fluid/Services/NemotronProvider.swift`](https://github.com/altic-dev/FluidVoice/blob/main/Sources/Fluid/Services/NemotronProvider.swift). It serves as the primary interface for integrating NVIDIA's Nemotron 3.5 speech models into macOS applications, handling everything from model downloading and caching to real-time audio stream processing.

## Core Features of NemotronProvider

### Multiple Operation Modes

The provider supports three distinct transcription modes defined in the `Mode` enum:

- **`.offline`** – High-accuracy batch transcription for recorded audio files
- **`.streaming`** – Ultra-low-latency real-time transcription for live audio
- **`.streaming320`** – Streaming pipeline using an alternative model folder configuration

Each mode exposes specific `displayName` and `folderHint` properties that determine model selection and loading behavior.

### Apple Silicon Hardware Optimization

The provider is compiled exclusively for `arm64` architecture using conditional compilation guards (`#if arch(arm64)`). On non-Apple-Silicon devices, the class automatically sets `isAvailable = false` and throws descriptive errors explaining the hardware requirement. This ensures optimal performance by targeting the Neural Engine and GPU compute units available on Apple Silicon chips.

### Automatic Model Download and Caching

On first initialization, `NemotronProvider` automatically downloads required CoreML artifacts from the Hugging Face repository `BarathwajAnandan/…` via `HuggingFaceModelDownloader`. The `prepare(progressHandler:)` method validates cached artifacts using `artifactsAreComplete(at:)` before loading, checking metadata, file size, and markup filtering to prevent corrupt downloads.

### Dynamic Compute Unit Selection

The provider attempts to load models on the Neural Engine first (`cpuAndNeuralEngine`). If initialization fails, it automatically falls back to `cpuAndGPU` via the `shouldRetryWithoutNeuralEngine(_:)` method. This fallback mechanism ensures transcription availability across different device capabilities while prioritizing energy-efficient processing.

### Language Selection Support

Users can select from approximately 40 supported locales via `SettingsStore.NemotronLanguage`. The provider applies language settings dynamically through `applySelectedLanguage(to:)`, updating the CoreML manager only when `SettingsStore.shared.selectedNemotronLanguage` changes, avoiding unnecessary model reloads.

### Streaming and Batch Transcription

**Streaming transcription** tracks processed audio via `streamedSampleCount` and feeds only delta buffers to `NemotronStreamingAsrManager` through `consumeDelta(from:manager:)`.

**Batch transcription** implements intelligent audio chunking for long files, splitting on energy-quiet boundaries using `chunkEnd(in:offset:)` and `quietestBoundary(...)` to avoid cutting mid-speech.

### Performance Profiling and Debugging

Optional component profiling records timing data for each model stage (pre-processor, encoder, decoder) when `ASRComponentProfilingEnabled` is set. Methods `startComponentProfilingIfNeeded`, `finishComponentProfilingIfNeeded`, and `logComponentProfile` provide detailed performance metrics for optimization.

## Implementation Architecture

### Audio Processing Pipeline

All input audio undergoes resampling to 16 kHz mono Float32 format via `resampleBuffer(_:targetSampleRate:)` before reaching the Nemotron models. This standardization ensures consistent inference regardless of source audio characteristics.

### Error Handling and Recovery

The provider implements robust error management:

- Corrupt cached artifacts (detected via HTML markup filtering) trigger automatic re-download
- Transcription failures in `transcribeSinglePass(_:)` reset the manager and clear internal counters
- The `clearCache()` method forces complete model re-download for debugging purposes

### Key Source Files

| File | Purpose |
|------|---------|
| [`Sources/Fluid/Services/NemotronProvider.swift`](https://github.com/altic-dev/FluidVoice/blob/main/Sources/Fluid/Services/NemotronProvider.swift) | Core provider implementation |
| [`Sources/Fluid/Persistence/SettingsStore.swift`](https://github.com/altic-dev/FluidVoice/blob/main/Sources/Fluid/Persistence/SettingsStore.swift) | Language selection persistence |
| `Sources/Fluid/Persistence/SettingsStore+NemotronLanguage.swift` | Supported locale definitions |
| [`Sources/Fluid/Services/ThinkingParsers.swift`](https://github.com/altic-dev/FluidVoice/blob/main/Sources/Fluid/Services/ThinkingParsers.swift) | Thinking flag handling for Nemotron models |

## Public API and Usage Examples

### Initialization and Model Preparation

```swift
import Fluid

// Initialize with streaming mode for real-time transcription
let nemotron = NemotronProvider(mode: .streaming)

// Or use offline mode for maximum accuracy
let offlineProvider = NemotronProvider(mode: .offline)

// Prepare downloads and loads models automatically
await nemotron.prepare { progress in
    switch progress {
    case .downloading(let fraction):
        print("Downloading... \(Int(fraction * 100))%")
    case .loading:
        print("Loading CoreML models")
    default: break
    }
}

```

### Batch Transcription

```swift
// Process a Float array of 16kHz PCM samples
let samples: [Float] = // ... audio data
let result = try await nemotron.transcribe(samples)
print("Transcribed text: \(result.text)")

```

### Streaming Transcription

```swift
var streamingProvider = NemotronProvider(mode: .streaming)
await streamingProvider.prepare()

// Feed audio incrementally
while let nextChunk = getNextAudioChunk() {
    let streamingResult = try await streamingProvider.transcribeStreaming(nextChunk)
    if !streamingResult.text.isEmpty {
        print("Partial: \(streamingResult.text)")
    }
}

```

### File-Based Transcription

```swift
let fileURL = URL(fileURLWithPath: "/path/to/audio.wav")
let fileResult = try await nemotron.transcribeFile(at: fileURL)
print("Full transcription: \(fileResult.text)")

```

### Cache Management

```swift
// Force re-download for debugging or corruption recovery
try await nemotron.clearCache()

```

## Summary

- **NemotronProvider** implements the `TranscriptionProvider` protocol in [`Sources/Fluid/Services/NemotronProvider.swift`](https://github.com/altic-dev/FluidVoice/blob/main/Sources/Fluid/Services/NemotronProvider.swift) to provide native NVIDIA Nemotron 3.5 support in FluidVoice.
- The class offers three operation modes—**offline**, **streaming**, and **streaming320**—catering to different latency and accuracy requirements.
- **Apple Silicon exclusivity** ensures optimal Neural Engine utilization, with automatic fallback to GPU when necessary.
- **Automatic model management** handles downloading, caching, and validation from Hugging Face repositories without manual intervention.
- The API provides **streaming-aware delta processing** and **smart audio chunking** for efficient handling of both real-time and batch transcription tasks.

## Frequently Asked Questions

### What hardware is required to use NemotronProvider?

NemotronProvider requires Apple Silicon (ARM64) architecture. The implementation uses `#if arch(arm64)` compilation guards to ensure the class only compiles on compatible devices. On Intel-based Macs, `isAvailable` returns false and the provider cannot be instantiated, as the CoreML models are optimized specifically for the Neural Engine and GPU compute units found in Apple Silicon chips.

### How does NemotronProvider handle model downloads and storage?

The provider automatically downloads CoreML artifacts from Hugging Face (specifically the `BarathwajAnandan/…` repository) on first use via `HuggingFaceModelDownloader`. Downloads are cached locally and validated before reuse through `artifactsAreComplete(at:)`, which checks file integrity, metadata consistency, and filters out HTML markup that might indicate download errors. Users can force re-download using the `clearCache()` method.

### What is the difference between streaming and offline modes?

**Offline mode** processes complete audio files with maximum accuracy using the full Nemotron 3.5 model, suitable for transcription of recorded meetings or podcasts. **Streaming mode** provides ultra-low-latency transcription by processing audio deltas incrementally, tracking processed samples via `streamedSampleCount` and feeding only new audio to `NemotronStreamingAsrManager`. **Streaming320** uses the same streaming pipeline but loads models from an alternative folder configuration optimized for specific latency requirements.

### How many languages does NemotronProvider support?

The provider supports approximately 40 languages and locales defined in `SettingsStore+NemotronLanguage.swift`. Users select their preferred language through `SettingsStore.shared.selectedNemotronLanguage`, which the provider applies via `applySelectedLanguage(to:)` to configure the underlying CoreML manager. The language setting persists across app launches and only triggers model reconfiguration when actually changed.