# How to Use the Audio Transcription API Across Different Providers in aisuite

> Transcribe audio using the aisuite API with OpenAI, Deepgram, or Google. Integrate seamlessly with automatic parameter translation and consistent results.

- Repository: [Andrew Ng/aisuite](https://github.com/andrewyng/aisuite)
- Tags: how-to-guide
- Published: 2026-06-15

---

**Initialize the `aisuite.Client` with your provider configurations, then call `client.audio.transcriptions.create()` using a `provider:model` string to transcribe audio files through OpenAI, Deepgram, or Google with automatic parameter translation and consistent result formats.**

The aisuite library from Andrew Ng provides a unified abstraction layer for audio transcription services, eliminating the need to learn disparate SDKs for each provider. By implementing a provider-agnostic interface in [`aisuite/client.py`](https://github.com/andrewyng/aisuite/blob/main/aisuite/client.py), the library routes requests to OpenAI Whisper, Deepgram Nova, or Google Speech-to-Text while normalizing parameters and results. This guide demonstrates how to use the audio transcription API across different providers in aisuite using the actual source code implementation and architectural patterns found in the repository.

## The Unified Audio Transcription Interface

At the core of aisuite's audio capabilities is the `Audio` class exposed through `Client.audio`, which provides a `Transcriptions` subclass for speech-to-text operations. Rather than instantiating provider-specific clients, you interact with a single method signature regardless of the backend service.

The `create` method in [`aisuite/client.py`](https://github.com/andrewyng/aisuite/blob/main/aisuite/client.py) accepts a **model identifier string** in the format `provider:model_name`, such as `openai:whisper-1` or `deepgram:nova-2`. This string determines which provider implementation handles the request, while the method signature remains constant across all providers:

```python
result = client.audio.transcriptions.create(
    model="openai:whisper-1",
    file="path/to/audio.wav",
    language="en",
    prompt="Custom vocabulary context"
)

```

All providers return a unified `TranscriptionResult` data structure defined in [`aisuite/framework/message.py`](https://github.com/andrewyng/aisuite/blob/main/aisuite/framework/message.py), ensuring that downstream code processes results identically regardless of whether OpenAI, Deepgram, or Google generated the transcription.

## Parameter Validation and Provider Mapping

Before invoking a provider, aisuite validates and transforms parameters using the `ParamValidator` class located in [`aisuite/framework/asr_params.py`](https://github.com/andrewyng/aisuite/blob/main/aisuite/framework/asr_params.py). This component handles three critical tasks:

- **Mapping common parameters** to provider-specific names (e.g., converting `language` to `language_code` for Google)
- **Validating against provider whitelists** defined in `PROVIDER_PARAMS` to prevent invalid API calls
- **Applying extra parameter modes** (`strict`, `warn`, or `permissive`) to handle unknown keys

The `extra_param_mode` configuration determines how the client handles parameters not recognized by the target provider. In `strict` mode, unknown parameters raise a `ValueError` immediately. In `warn` mode, the client logs a warning but proceeds. In `permissive` mode, all parameters pass through to the provider API unchecked.

The mapping logic resides in `ParamValidator._transform_value`, which consults provider-specific dictionaries to translate standard AISuite parameter names into the format required by each underlying API.

## Provider-Specific Implementations

The `ProviderFactory` in [`aisuite/provider.py`](https://github.com/andrewyng/aisuite/blob/main/aisuite/provider.py) instantiates concrete provider classes (`OpenaiProvider`, `DeepgramProvider`, `GoogleProvider`) based on the model prefix. Each provider implements an `Audio` subclass with `Transcriptions` methods that handle API-specific payload construction.

### OpenAI Provider

Located in [`aisuite/providers/openai_provider.py`](https://github.com/andrewyng/aisuite/blob/main/aisuite/providers/openai_provider.py), the OpenAI implementation forwards requests directly to the native `client.audio.transcriptions.create` method. Parameter mapping remains largely one-to-one, with common parameters like `language`, `prompt`, and `temperature` passing through unchanged.

Key provider-specific parameters include `response_format`, `timestamp_granularities`, and `stream` for enabling real-time transcription.

### Deepgram Provider

The Deepgram implementation in [`aisuite/providers/deepgram_provider.py`](https://github.com/andrewyng/aisuite/blob/main/aisuite/providers/deepgram_provider.py) prepares raw byte payloads for the Deepgram SDK v5. While the `language` parameter maps directly, Deepgram offers unique capabilities accessible through provider-specific parameters:

- `punctuate` for automatic punctuation
- `diarize` for speaker identification
- `smart_format` for automatic formatting
- `interim_results` for streaming partial transcripts

### Google Provider

The Google provider in [`aisuite/providers/google_provider.py`](https://github.com/andrewyng/aisuite/blob/main/aisuite/providers/google_provider.py) constructs a `RecognitionConfig` object for the Google Speech-to-Text API. This requires the most aggressive parameter mapping:

- `language` transforms to `language_code` using the `GOOGLE_LANGUAGE_MAP` to expand two-letter codes (e.g., `en` becomes `en-US`)
- `prompt` becomes `speech_contexts` wrapped as JSON
- Standard parameters map to `encoding`, `sample_rate_hertz`, and `enable_automatic_punctuation`

## Batch and Streaming Transcription Modes

AISuite supports both synchronous batch processing and asynchronous streaming transcription across compatible providers.

### Batch Transcription

The standard `create` method processes entire audio files and returns complete `TranscriptionResult` objects containing the full text, segments, and metadata. This mode works identically across all providers:

```python
result = client.audio.transcriptions.create(
    model="google:gemini-1.5-flash",
    file="audio.flac",
    language="en",
    enable_word_time_offsets=True
)
print(result.text)
print(result.words)  # Available when requested

```

### Streaming Implementation

For real-time transcription, providers implement `create_stream_output`, an async generator yielding `StreamingTranscriptionChunk` objects. When you pass `stream=True`, the client routes to the streaming implementation instead of the batch method.

OpenAI, Deepgram, and Google all support streaming through this unified interface:

```python
async for chunk in client.audio.transcriptions.create_stream_output(
    model="openai:whisper-1",
    file="live_audio.wav",
    language="en",
    stream=True
):
    print(f"{'FINAL' if chunk.is_final else 'PART'}: {chunk.text}")

```

## Complete Implementation Examples

### Basic Batch Transcription Across Providers

Configure multiple providers in a single client instance and switch between them by changing the model string:

```python
from aisuite import Client

client = Client(
    provider_configs={
        "openai": {"api_key": "sk-..."},
        "deepgram": {"api_key": "DG..."},
        "google": {
            "project_id": "my-gcp-proj",
            "region": "us-central1",
            "application_credentials": "/path/to/key.json",
        },
    }
)

# OpenAI Whisper

result = client.audio.transcriptions.create(
    model="openai:whisper-1",
    file="speech.mp3",
    language="en",
)
print(result.text)

# Deepgram Nova-2 with provider-specific features

result = client.audio.transcriptions.create(
    model="deepgram:nova-2",
    file="speech.wav",
    language="en",
    punctuate=True,
    diarize=True,
)
print(result.segments)

# Google with language code expansion

result = client.audio.transcriptions.create(
    model="google:gemini-1.5-flash",
    file="speech.flac",
    language="en",  # Auto-expands to "en-US"

    prompt="medical terminology",
)
print(result.words)

```

### Streaming Transcription with Async Iteration

```python
import asyncio
from aisuite import Client

async def stream_transcription():
    client = Client(provider_configs={"openai": {"api_key": "sk-..."}})
    async for chunk in client.audio.transcriptions.create_stream_output(
        model="openai:whisper-1",
        file="long_speech.wav",
        language="en",
        stream=True,
    ):
        print(f"{'FINAL' if chunk.is_final else 'PART'}: {chunk.text}")

asyncio.run(stream_transcription())

```

### Controlling Parameter Validation Strictness

```python

# Strict mode rejects unknown parameters

client_strict = Client(extra_param_mode="strict")
try:
    client_strict.audio.transcriptions.create(
        model="openai:whisper-1",
        file="speech.mp3",
        unknown_param=123,
    )
except ValueError as e:
    print(f"Strict mode caught: {e}")

# Permissive mode passes unknown parameters to the API

client_perm = Client(extra_param_mode="permissive")
result = client_perm.audio.transcriptions.create(
    model="openai:whisper-1",
    file="speech.mp3",
    unknown_param=123,  # Sent to OpenAI API without error

)

```

## Error Handling and Result Normalization

Provider-specific errors are wrapped as `ASRError` exceptions (or `LLMError` for chat operations) to present a consistent exception interface. This allows you to catch transcription failures uniformly regardless of whether the underlying error originated from OpenAI's rate limits, Deepgram's authentication issues, or Google's quota restrictions.

The `TranscriptionResult` object in [`aisuite/framework/message.py`](https://github.com/andrewyng/aisuite/blob/main/aisuite/framework/message.py) normalizes response structures across providers. While OpenAI returns simple text objects and Deepgram returns complex segment arrays, aisuite exposes these through standardized attributes like `result.text`, `result.segments`, and `result.words` where supported.

## Summary

- **Unified Interface**: Use `client.audio.transcriptions.create()` with a `provider:model` string to access OpenAI, Deepgram, or Google transcription services through identical method signatures.
- **Automatic Parameter Mapping**: The `ParamValidator` in [`aisuite/framework/asr_params.py`](https://github.com/andrewyng/aisuite/blob/main/aisuite/framework/asr_params.py) translates common parameters like `language` to provider-specific names (e.g., `language_code` for Google) and validates against provider whitelists.
- **Flexible Validation Modes**: Configure `extra_param_mode` as `strict`, `warn`, or `permissive` to control how the client handles unknown parameters.
- **Streaming Support**: Use `create_stream_output()` with `stream=True` for real-time transcription, yielding `StreamingTranscriptionChunk` objects across all supported providers.
- **Consistent Results**: All providers return `TranscriptionResult` objects defined in [`aisuite/framework/message.py`](https://github.com/andrewyng/aisuite/blob/main/aisuite/framework/message.py), normalizing text, segments, and timing information into a standard format.

## Frequently Asked Questions

### How do I switch between transcription providers without changing my code?

Change the `model` parameter string from `provider:model_name` to target a different backend. For example, changing `openai:whisper-1` to `deepgram:nova-2` routes the request through Deepgram instead of OpenAI, while keeping the same `client.audio.transcriptions.create()` call and result handling code. Ensure the target provider is configured in your `Client` initialization.

### What is the difference between strict and permissive parameter modes?

**Strict mode** raises a `ValueError` immediately when you pass parameters not recognized by the provider's whitelist, preventing accidental API errors. **Permissive mode** passes all unknown parameters through to the underlying API, allowing access to beta features or provider-specific parameters not yet mapped in aisuite. **Warn mode** (the default) logs a warning but continues execution.

### How does streaming transcription work across different providers?

Streaming uses the async `create_stream_output()` method, which returns an async generator yielding `StreamingTranscriptionChunk` objects. Each chunk contains partial text and an `is_final` boolean indicating whether the segment is complete. OpenAI, Deepgram, and Google all implement this interface, though the underlying streaming protocols differ significantly between providers.

### Why does Google Speech-to-Text convert my two-letter language codes?

The Google provider in [`aisuite/providers/google_provider.py`](https://github.com/andrewyng/aisuite/blob/main/aisuite/providers/google_provider.py) automatically expands standard two-letter codes (like `en` or `es`) to Google-specific regional codes (like `en-US` or `es-ES`) using the `GOOGLE_LANGUAGE_MAP` dictionary. This ensures compatibility with Google's API requirements while allowing you to use standard language codes consistently across all providers in your aisuite code.