aisuite Chat Completions API vs Agents API: Core Differences and When to Use Each

The Chat Completions API is a thin, stateless wrapper around provider-specific chat endpoints, while the Agents API adds a persistent orchestration layer with state management, policy enforcement, and structured tracing on top of the same underlying completion logic.

If you are building with the andrewyng/aisuite library, you can choose between two high-level patterns for interacting with language models. Understanding the architectural split between the Chat Completions API and the Agents API helps you decide whether you need a simple one-off call or a long-running autonomous workflow.

Chat Completions API: Direct, Stateless Requests

The Chat Completions API is designed for direct, caller-controlled interaction. Its entry point is Client.chat.completions.create(...) (and the async variant acreate), implemented in aisuite/client.py across lines 85–99, 112–119, and 144–184.

Entry Points and Tool Loops

When you invoke client.chat.completions.create(), you pass a model string, a messages list, and optional arguments such as tools or max_turns. Any tool-call loop is handled internally by _tool_runner and _atool_runner in aisuite/client.py (lines 332–382 and 383–444). Because this path is stateless per call, the library does not preserve conversation history between separate create() calls unless you explicitly pass it back in the messages parameter.

Streaming and max_turns

You can enable streaming by setting stream=True, which triggers the _prepare_stream_kwargs helper and returns an iterator of OpenAI-shaped chunks. However, as implemented in aisuite/client.py, streaming cannot be combined with max_turns. If you need multi-turn tool execution, you must accept a non-streaming response.

Agents API: Stateful Autonomous Orchestration

The Agents API builds on the same chat completion machinery but wraps it inside a higher-level execution engine. Its primary entry point is aisuite.agents.runner.AgentRunner, exposed through the Agent dataclass in aisuite/agents/types.py.

Persistent State and Context

Unlike the Chat Completions path, an agent maintains run-level state across invocations. The AgentRunner manages an AgentContext object (defined in aisuite/agents/context.py), a StateStore, and an ArtifactStore (specified in aisuite/agents/state_store.py). These components automatically update after each turn, so your agent can remember past interactions, store intermediate results, and resume workflows without manually reconstructing the messages list.

Policy Engine and Observability

Every agent can enforce a tool-use policy through the dedicated policy engine in aisuite/agents/policies.py. This is distinct from the optional tool_policy dict you might pass to the Chat Completions API; the Agents API applies its configured policy automatically on every tool call.

For observability, the Chat Completions layer emits low-level events such as model.send, model.response, and model.error. The Agents API enriches this with higher-level trace events—including agent.run.start, agent.run.end, tool.policy, and tool.execution—via the tracing subsystem in aisuite/tracing.

Side-by-Side Comparison

• Purpose: The Chat Completions API issues single or multi-turn chat requests where you control the flow. The Agents API encapsulates a full-featured autonomous agent that persists state and manages its own run context.
• Entry points: Client.chat.completions.create(...) in aisuite/client.py versus AgentRunner in aisuite/agents/runner.py.
• State handling: Chat Completions is stateless per call. Agents maintain persistent context through AgentContext, StateStore, and ArtifactStore.
• Policy enforcement: Chat Completions accepts an optional tool_policy forwarded to the internal runner. Agents use the built-in policy engine in aisuite/agents/policies.py that is automatically applied each turn.
• Extensibility: You can supply custom tools to either API, but agents can be composed with toolkits, custom artifact stores (e.g., PostgreSQL), and per-agent configuration in aisuite/agents/context.py.
• Tracing: Chat Completions emits model-level events. Agents emit richer lifecycle events through aisuite/tracing.

Practical Code Examples

Stateless Chat Completion with a Tool Loop

from aisuite import Client

client = Client(provider_configs={"openai": {"api_key": "sk-*****"}})

# Simple, stateless chat request

resp = client.chat.completions.create(
    model="openai:gpt-4o-mini",
    messages=[{"role": "user", "content": "What is the capital of France?"}],
)
print(resp.choices[0].message.content)

# Multi-turn tool execution (max_turns = 3)

def get_current_time():
    import datetime
    return {"time": datetime.datetime.utcnow().isoformat()}

resp = client.chat.completions.create(
    model="openai:gpt-4o-mini",
    messages=[{"role": "user", "content": "What time is it now?"}],
    tools=[get_current_time],
    max_turns=3,
)
print(resp.choices[0].message.content)

Stateful Agent Runner

from aisuite.agents.types import Agent
from aisuite.agents.runner import AgentRunner

def get_current_time():
    import datetime
    return {"time": datetime.datetime.utcnow().isoformat()}

my_agent = Agent(
    name="time-assistant",
    tools=[get_current_time],
    policy={"allow": ["get_current_time"]}
)

runner = AgentRunner(agent=my_agent)

# The runner persists context and state across turns automatically

run_result = runner.run(
    model="openai:gpt-4o-mini",
    messages=[{"role": "user", "content": "Tell me the current UTC time."}]
)

print(run_result.final_response.choices[0].message.content)

Summary

• The Chat Completions API in aisuite/client.py provides a thin, stateless wrapper around provider endpoints, with optional tool loops via _tool_runner and streaming via _prepare_stream_kwargs.
• The Agents API layers persistent state, policy enforcement, and structured tracing on top of the same completion logic through AgentRunner in aisuite/agents/runner.py.
• Use Chat Completions for one-off or caller-managed conversations.
• Use Agents for long-running workflows that require autonomous context management, reusable state stores, and automatic policy checks.

Frequently Asked Questions

Can I use streaming with the Agents API?

Streaming is technically possible within the Agents API, but the AgentRunner manages the full turn cycle internally. In practice, you typically call the non-streaming mode and let the agent handle any internal streaming, because the runner must inspect and act on each tool call before proceeding.

Is the Chat Completions API completely stateless?

Yes. Every call to Client.chat.completions.create(...) is independent; if you want the model to remember earlier turns, you must manually maintain and pass the messages list on each request. There is no server-side or library-side conversation memory between calls.

How does policy enforcement differ between the two APIs?

With the Chat Completions API, you can pass an optional tool_policy dictionary that is forwarded to the internal Tools runner. With the Agents API, policy enforcement is first-class: the aisuite.agents.policies.py engine validates and filters tool calls automatically on every execution turn based on the agent's configuration.

Which API should I use for long-running workflows?

Choose the Agents API for long-running or recurring workflows. Its AgentContext, StateStore, and ArtifactStore (all defined in aisuite/agents/context.py and aisuite/agents/state_store.py) keep track of history and artifacts across runs, while the built-in policy engine and tracing subsystem provide governability and observability that the stateless Chat Completions path does not offer.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}