aisuite Chat vs. Audio APIs: Key Differences and Usage Guide

aisuite's Chat API handles text-based conversational completions with tool execution and streaming, while the Audio API provides speech-to-text transcription with optional provider-specific streaming.

The aisuite library, developed by Andrew Ng's team, offers a unified interface for multiple AI providers through its Client class. The library exposes two distinct high-level API groups—client.chat for text completions and client.audio for speech transcription—that differ fundamentally in purpose, architecture, and capabilities.

Core API Entry Points

Both APIs follow a consistent naming convention but serve entirely different use cases.

API	Entry Point	Return Type
Chat	client.chat.completions.create(...)	ChatCompletionResponse or streaming chunks
Audio	client.audio.transcriptions.create(...)	TranscriptionResponse

The Chat class is defined at lines 15-24 in aisuite/client.py [source], while the Audio class with its inner Transcriptions component appears at lines 36-48 [source].

Feature Comparison: Chat API vs. Audio API

Purpose and Scope

client.chat is designed for complex conversational workflows. It supports multi-turn dialogues, tool-augmented reasoning, and streaming responses. The API can execute functions automatically and inject results back into the conversation context.

client.audio focuses exclusively on converting audio into text. It does not maintain conversation state or handle tool execution—each transcription request is independent and stateless.

Tool Integration Capabilities

The Chat API implements a full tool execution framework. The _tool_runner and _atool_runner methods (lines 32-78 in aisuite/client.py [source]) parse tool calls, execute them synchronously or asynchronously, and manage the conversation loop for up to max_turns iterations.

The Audio API has no tool handling whatsoever. The Transcriptions.create method simply forwards requests to the provider's audio endpoint without any intermediate processing.

Streaming Support Differences

API	Streaming Method	Constraints
Chat	Provider's chat_completions_create_stream	Cannot combine with max_turns or tool loops [source]
Audio	Provider-specific create_stream_output	Optional; raises clear error if unavailable [source]

Tracing and Observability

Chat API calls emit detailed trace events through aisuite's tracing subsystem. Events include model.send, model.response, and model.error (lines 35-52 [source]), enabling comprehensive logging and debugging.

The Audio API provides no tracing integration—requests pass directly to the provider without event emission.

Parameter Validation

• Chat: Uses generic parameter validation; strips tool-related kwargs (tools, max_turns, tool_policy) before provider submission
• Audio: Employs ParamValidator from aisuite/framework/asr_params.py to map common transcription parameters (language, prompt, temperature) while forwarding provider-specific kwargs [source]

Provider Resolution Architecture

Both APIs share a common provider resolution pattern through the model string format provider:model.

The Chat._resolve_provider method (lines 52-70 [source]) lazily instantiates providers as needed. The Transcriptions.create method replicates this logic but adds a specific check for the audio attribute on the resolved provider (lines 71-84 [source]).

Practical Code Examples

Basic Chat Completion

from aisuite import Client

client = Client()

# Single-shot text completion

result = client.chat.completions.create(
    model="openai:gpt-4o-mini",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Explain the difference between cats and dogs."},
    ],
    temperature=0.7,
)
print(result.choices[0].message.content)

Chat with Tool Execution

# Multi-turn tool-augmented conversation

result = client.chat.completions.create(
    model="openai:gpt-4o-mini",
    messages=[{"role": "user", "content": "List files in the current directory."}],
    tools=[my_shell_tool],      # Callable conforming to Tools API

    max_turns=2,                # Allow up to 2 tool execution rounds

)

# Result includes intermediate tool messages and final answer

Audio Transcription (Batch)

# Standard speech-to-text conversion

transcript = client.audio.transcriptions.create(
    model="openai:whisper-1",
    file="speech.mp3",
    language="en",
    prompt="Transcribe the meeting notes.",
)
print(transcript.text)   # Unified text field across all providers

Audio with Provider-Specific Streaming

# Streaming transcription (provider-dependent implementation)

stream = client.audio.transcriptions.create(
    model="deepgram:nova-2",
    file="lecture.wav",
    stream=True,
    punctuate=True,        # Deepgram-specific parameter

)
for chunk in stream:
    print(chunk.text, end="")

When to Use Each API

Choose client.chat when you need:

• Conversational AI agents
• Multi-step reasoning with tool calling
• Streaming text generation
• Full observability through tracing

Choose client.audio when you need:

• Speech-to-text conversion
• Audio content ingestion pipelines
• Voice-controlled application inputs
• Minimal-overhead transcription

Summary

• Chat API (aisuite/client.py lines 15-78) provides a full-featured, stateful conversational interface with tool execution, multi-turn loops, and comprehensive tracing
• Audio API (aisuite/client.py lines 36-48, 55-78) offers a lightweight, stateless transcription wrapper focused solely on speech-to-text conversion
• Both APIs use identical provider:model syntax but differ in parameter validation, streaming constraints, and observability features
• Streaming in Chat excludes tool execution; streaming in Audio depends entirely on provider implementation

Frequently Asked Questions

Can I use tool execution with streaming in the Chat API?

No. According to the source code in aisuite/client.py (lines 100-108), streaming and tool loops are mutually exclusive. The max_turns parameter for multi-turn tool execution requires synchronous processing, while streaming returns chunks immediately without intermediate tool result injection.

Does the Audio API support real-time streaming transcription?

Provider-dependent. The create_stream_output method is optional—some providers implement it, others don't. If unavailable, aisuite raises a clear error rather than failing silently (lines 47-64 in aisuite/client.py).

How does aisuite handle authentication differences across providers?

Both APIs use the same underlying provider resolution mechanism. The Client class lazily instantiates provider clients based on environment variables or explicit configuration, abstracting provider-specific authentication into the provider:model string format.

Can I convert audio to text and then feed it into a chat completion?

Yes—this is a common pattern. Use client.audio.transcriptions.create() to obtain text, then pass that text as a user message to client.chat.completions.create(). The two APIs are designed to compose cleanly, though they maintain no internal connection.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}