# How to Implement Audio Transcription with the Audio Class in aisuite

> Effortlessly transcribe audio using the aisuite Audio class. Integrate OpenAI, Google, or Deepgram with a single line of code and a provider model string for seamless speech-to-text.

- Repository: [Andrew Ng/aisuite](https://github.com/andrewyng/aisuite)
- Tags: how-to-guide
- Published: 2026-07-28

---

**Call `client.audio.transcriptions.create()` with a `"provider:model"` string and an audio file path to transcribe speech using any supported provider like OpenAI, Google, or Deepgram without changing your application code.**

The aisuite library provides a unified, provider-agnostic interface for audio transcription tasks. By using the `Audio` class exposed through `Client.audio`, you can implement speech-to-text functionality across multiple providers including OpenAI Whisper, Google Cloud Speech-to-Text, and Deepgram. This article explains how to implement audio transcription with the Audio class in aisuite using actual source code patterns from the `andrewyng/aisuite` repository.

## Understanding the Audio API Architecture

The Audio API consists of three distinct layers that abstract provider complexity while maintaining flexibility.

### The Client Façade Layer

In [`aisuite/client.py`](https://github.com/andrewyng/aisuite/blob/main/aisuite/client.py) (around lines 36-48), the `Audio` class acts as the entry point. The `Client` exposes an `audio` attribute that returns this `Audio` instance, which in turn exposes a `transcriptions` sub-interface. When you call `client.audio.transcriptions.create()`, the client handles model string parsing, parameter validation, and provider delegation automatically.

### The Provider Abstraction

The abstract contract is defined in [`aisuite/provider.py`](https://github.com/andrewyng/aisuite/blob/main/aisuite/provider.py) (around lines 26-45). Here, the base `Audio` class defines the structure that all providers must implement, including the nested `Transcription` class that specifies the `create` and `create_stream_output` method signatures. This ensures every provider implements the same interface for batch and streaming transcription.

### Provider-Specific Implementations

Concrete implementations reside in provider-specific files. For example, [`aisuite/providers/openai_provider.py`](https://github.com/andrewyng/aisuite/blob/main/aisuite/providers/openai_provider.py) contains the `OpenAIAudio.Transcriptions` class that implements the actual API calls to OpenAI's Whisper model. Similarly, [`aisuite/providers/deepgram_provider.py`](https://github.com/andrewyng/aisuite/blob/main/aisuite/providers/deepgram_provider.py) implements `DeepgramAudio.Transcriptions` for Deepgram's speech-to-text API.

## Prerequisites and Setup

Install aisuite with the specific provider extras you intend to use:

```bash
pip install "aisuite[openai]"  # or [google], [deepgram], etc.

```

Configure credentials via environment variables (e.g., `OPENAI_API_KEY`) or pass them explicitly through the `provider_configs` dictionary when initializing the `Client`.

## Implementing Batch Audio Transcription

For standard transcription tasks where you process an entire audio file and receive the complete text, use the batch method. The `model` parameter uses the format `"provider:model"` (e.g., `"openai:whisper-1"`), which the `Client` validates and routes via `ProviderFactory`.

```python
from aisuite import Client

# Initialize client - API key can be loaded from env var OPENAI_API_KEY

client = Client(provider_configs={"openai": {}})

# Transcribe audio file

result = client.audio.transcriptions.create(
    model="openai:whisper-1",
    file="speech.mp3",          # Can be a file path or file-like object

    language="en",              # Common ASR parameter

    prompt="Transcribe the meeting notes clearly."
)

print(result.text)  # result is a TranscriptionResponse from aisuite/framework/message.py

```

The `ParamValidator` in [`aisuite/framework/asr_params.py`](https://github.com/andrewyng/aisuite/blob/main/aisuite/framework/asr_params.py) normalizes common parameters like `language`, `prompt`, and `temperature` before forwarding the request to the provider's concrete implementation.

## Implementing Streaming Audio Transcription

For real-time processing or large files where you want incremental results, pass `stream=True` to receive chunks as they become available:

```python
client = Client(provider_configs={"openai": {}})

# Stream chunks of transcription as they arrive

for chunk in client.audio.transcriptions.create(
    model="openai:whisper-1",
    file="lecture.wav",
    stream=True,                # Request streaming output

    response_format="verbose_json"
):
    # chunk is a StreamingTranscriptionChunk

    print(chunk.text, end="")

```

When `stream=True`, the client recognizes the flag and invokes the provider's `create_stream_output` method instead of the standard `create` method. This allows providers like Deepgram or OpenAI to stream partial results back to your application.

## Working with Alternative Providers

Switching providers requires only changing the model string and configuration. The `Client` automatically loads the appropriate provider class via `ProviderFactory.create_provider` based on the prefix before the colon.

```python
client = Client(
    provider_configs={
        "deepgram": {"api_key": "YOUR_DEEPGRAM_KEY"}
    }
)

result = client.audio.transcriptions.create(
    model="deepgram:nova-2",
    file="interview.mp3",
    language="en",
    punctuate=True,   # Deepgram-specific parameter

    diarize=True      # Deepgram-specific parameter

)

print(result.text)

```

Provider-specific kwargs like `punctuate` and `diarize` are passed through untouched after the `ParamValidator` extracts and validates common ASR parameters. This happens in [`aisuite/framework/asr_params.py`](https://github.com/andrewyng/aisuite/blob/main/aisuite/framework/asr_params.py), ensuring unknown parameters don't break the request while still allowing provider-native features.

## Parameter Validation and Error Handling

Control how the client handles unknown or extra parameters using the `extra_param_mode` setting:

```python
client = Client(extra_param_mode="warn")  # Default behavior - logs warning

result = client.audio.transcriptions.create(
    model="openai:whisper-1",
    file="audio.mp3",
    foobar="ignored"  # Unknown parameter triggers warning but call proceeds

)

```

Set `extra_param_mode="strict"` to raise an exception on unknown keys, or `"permissive"` to allow them silently. The `ParamValidator.validate_and_map` method in [`aisuite/framework/asr_params.py`](https://github.com/andrewyng/aisuite/blob/main/aisuite/framework/asr_params.py) implements this logic, checking parameters against known ASR fields before delegation.

## Summary

- The `Audio` class in [`aisuite/client.py`](https://github.com/andrewyng/aisuite/blob/main/aisuite/client.py) provides a unified interface for transcription via `client.audio.transcriptions.create()`
- Model strings use the format `"provider:model"` (e.g., `"openai:whisper-1"`), parsed by `ProviderFactory` to load the correct provider implementation
- Batch transcription returns a `TranscriptionResponse` object containing the full text; streaming with `stream=True` yields `StreamingTranscriptionChunk` objects
- Provider implementations reside in files like [`aisuite/providers/openai_provider.py`](https://github.com/andrewyng/aisuite/blob/main/aisuite/providers/openai_provider.py) and [`aisuite/providers/deepgram_provider.py`](https://github.com/andrewyng/aisuite/blob/main/aisuite/providers/deepgram_provider.py), each implementing `Audio.Transcription.create()` and optionally `create_stream_output()`
- Parameters are validated by `ParamValidator` in [`aisuite/framework/asr_params.py`](https://github.com/andrewyng/aisuite/blob/main/aisuite/framework/asr_params.py) with configurable `extra_param_mode` settings for handling unknown keys

## Frequently Asked Questions

### What audio file formats does aisuite support?

aisuite delegates format handling to the underlying provider. OpenAI Whisper supports mp3, mp4, mpeg, mpga, m4a, wav, and webm. Deepgram supports over 40 formats including mp3, wav, and flac. The library passes your file object directly to the provider's API without intermediate conversion, so supported formats depend on your chosen provider's specifications.

### How do I switch between transcription providers?

Change the model string from `"openai:whisper-1"` to `"deepgram:nova-2"` or `"google:gemini-2.0-flash-exp"`. Ensure you have installed the appropriate provider extras (e.g., `pip install "aisuite[deepgram]"`) and configured the API credentials. The `Client` automatically instantiates the correct provider class via `ProviderFactory.create_provider` based on the prefix before the colon, requiring zero changes to your transcription logic.

### Can I use aisuite for real-time streaming transcription?

Yes, pass `stream=True` to `client.audio.transcriptions.create()`. This triggers the `create_stream_output` method in the provider's `Audio.Transcription` implementation. Providers like Deepgram return chunks as they become available through their streaming APIs, while others may buffer and simulate streaming depending on native capabilities. Each chunk returned is a `StreamingTranscriptionChunk` object as defined in [`aisuite/framework/message.py`](https://github.com/andrewyng/aisuite/blob/main/aisuite/framework/message.py).

### Where are provider-specific parameters validated?

The `ParamValidator` class in [`aisuite/framework/asr_params.py`](https://github.com/andrewyng/aisuite/blob/main/aisuite/framework/asr_params.py) normalizes common ASR parameters like `language`, `prompt`, and `temperature` across all providers. Unknown parameters are handled according to the `extra_param_mode` setting: `"warn"` logs a warning (default), `"strict"` raises an exception, and `"permissive"` allows them silently. Provider-specific parameters like Deepgram's `punctuate` or `diarize` are passed through untouched after common parameter extraction, ensuring you can access native provider features without wrapper limitations.