How to Configure Chat Completion Parameters Provider-Independently in aisuite

aisuite lets you pass standard completion arguments like temperature and max_tokens as keyword arguments to client.chat.completions.create(), automatically forwarding them to any supported LLM provider without changing your code.

The aisuite library by Andrew Ng provides a unified interface for interacting with multiple large language model providers. By implementing a standardized client API in andrewyng/aisuite, you can configure chat completion parameters provider-independently without modifying your application code when switching between models like GPT-4o, Llama 3.1, or Gemini 1.5 Flash.

How Provider-Independent Configuration Works

In aisuite/client.py, the Chat.create method serves as the central dispatch point. When you call client.chat.completions.create(), the method extracts the provider identifier and model name from the model string (e.g., "openai:gpt-4o"), then forwards all remaining kwargs directly to the concrete provider's chat_completions_create implementation.

# aisuite/client.py (simplified)

def create(self, model: str, messages: list, **kwargs):
    provider, model_name = self._resolve_provider(model)
    # Tool handling omitted for brevity

    response = provider.chat_completions_create(model_name, messages, **kwargs)
    return self._extract_thinking_content(response)

Each provider implements the abstract chat_completions_create method defined in aisuite/provider.py. The base Provider class establishes the contract, while concrete subclasses handle parameter translation:

• OpenAI (aisuite/providers/openai_provider.py): Forwards kwargs directly to the OpenAI SDK
• Hugging Face (aisuite/providers/huggingface_provider.py): Passes parameters to the HF inference client (TGI backend)
• Google Gemini (aisuite/providers/gemini_provider.py): Maps temperature to GenerationConfig and handles max_output_tokens

Standard Completion Parameters

You can pass these standard arguments to any provider using identical syntax:

• temperature: Sampling temperature (typically 0-2 range)
• max_tokens / max_output_tokens: Token generation limits
• top_p: Nucleus sampling parameter
• stop: Stop sequence tokens
• stream: Boolean to enable streaming responses

The unified message schema in aisuite/framework/message.py ensures that conversation history remains compatible across providers regardless of these configuration parameters.

Cross-Provider Configuration Examples

OpenAI and Hugging Face

Both providers accept identical parameter sets through the same API call:

from aisuite import Client

client = Client(provider_configs={
    "openai": {"api_key": "sk-..."},
    "huggingface": {"token": "hf_..."}
})

messages = [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "Explain quantum computing."}
]

# OpenAI - parameters forwarded directly

response = client.chat.completions.create(
    model="openai:gpt-4o-mini",
    messages=messages,
    temperature=0.7,
    max_tokens=256
)

# Hugging Face - same parameters, forwarded to TGI backend

response = client.chat.completions.create(
    model="huggingface:meta-llama/Meta-Llama-3.1-8B-Instruct",
    messages=messages,
    temperature=0.7,
    max_tokens=256
)

Google Gemini Mapping

The Gemini provider automatically translates standard parameters to Google's native format:

response = client.chat.completions.create(
    model="gemini:gemini-1.5-flash",
    messages=messages,
    temperature=0.7,          # Mapped to GenerationConfig(temperature=0.7)

    max_output_tokens=256     # Gemini-specific parameter name also supported

)

Streaming Responses

The stream parameter works provider-independently when the underlying provider supports the chat_completions_create_stream method defined in aisuite/provider.py:

stream = client.chat.completions.create(
    model="openai:gpt-4o-mini",
    messages=messages,
    temperature=0.5,
    stream=True
)

for chunk in stream:
    # Each chunk follows the OpenAI chat.completion.chunk shape

    print(chunk.choices[0].delta.content or "", end="", flush=True)

Handling Unsupported Parameters

If a specific provider does not support a parameter, the implementation safely ignores it rather than raising an error. According to the docstring in aisuite/client.py (line 677), temperature is currently only meaningful for OpenAI-style models, while other providers may ignore it. This forward-compatible design ensures your code remains stable when switching between models with different capabilities.

Summary

• Central dispatch: aisuite/client.py forwards all kwargs from Chat.create to provider-specific implementations without validation, enabling provider-independent configuration.
• Unified interface: Pass standard parameters (temperature, max_tokens, stream) regardless of whether you use OpenAI, Hugging Face, or Gemini.
• Automatic translation: Providers like Gemini map standard arguments to their native formats (e.g., GenerationConfig) internally.
• Safe fallback: Unsupported parameters are ignored rather than causing runtime errors, ensuring provider interoperability.

Frequently Asked Questions

Does aisuite validate chat completion parameters before sending them to providers?

No. According to the implementation in aisuite/client.py, the client extracts the provider and model from the model string, then forwards all remaining keyword arguments directly to the provider's chat_completions_create method. Parameter validation occurs at the provider level (e.g., OpenAI SDK or Gemini API), not within aisuite itself.

Can I use the same temperature setting across all providers in aisuite?

While you can pass temperature to any provider, not all models interpret it identically. The docstring in aisuite/client.py explicitly notes that temperature is currently only meaningful for OpenAI-style models. Providers like Hugging Face may forward it to the TGI backend, but Gemini maps it to GenerationConfig, and some providers may ignore unsupported parameters entirely.

How does aisuite handle provider-specific parameters like Gemini's max_output_tokens?

aisuite forwards all kwargs transparently. In aisuite/providers/gemini_provider.py, the implementation maps standard parameters like temperature to Google's GenerationConfig while also accepting native parameters like max_output_tokens. You can use either the OpenAI-style max_tokens or the Gemini-specific max_output_tokens, and the provider handles the translation internally.

Is streaming supported for all providers in aisuite?

Streaming is supported provider-independently when the underlying provider implements the chat_completions_create_stream method defined in aisuite/provider.py. You pass stream=True to client.chat.completions.create() regardless of the provider, and the response returns an iterator following the OpenAI chat.completion.chunk format. Check individual provider implementations to confirm streaming availability for specific models.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}