How aisuite Handles Intermediate Messages During Tool-Calling Conversations

aisuite tracks multi-turn tool-calling conversations by maintaining two parallel collections—intermediate_responses for raw model outputs and intermediate_messages for every utterance—and attaches both to the final response object returned from aisuite/client.py.

When a model request triggers external tools in the andrewyng/aisuite library, the framework must preserve every turn between the LLM and the tool so that downstream agents and UIs can reconstruct the full dialogue. This is handled inside a dedicated request loop in aisuite/client.py that accumulates each model reply and tool output before returning the final answer. Understanding how aisuite handles intermediate messages during tool-calling conversations is essential for building observable agents and debugging multi-step reasoning chains.

Parallel Collections That Store Intermediate Messages

The framework records the conversation using two synchronized lists that serve different downstream needs.

intermediate_responses

This list stores the raw Response objects returned by the model after each tool call. In aisuite/client.py, the list is populated inside the main request loop and then trimmed so that it excludes the final response:

response.intermediate_responses = intermediate_responses[:-1]

The slice [:-1] ensures that only the mid-conversation model replies are preserved, while the final answer remains the primary payload.

intermediate_messages

This list stores Message objects representing every utterance in the conversation, including user prompts, tool-generated outputs, and the model’s own replies. During request handling in aisuite/client.py, each model message is appended to the list:

intermediate_messages.append(response.choices[0].message)

After the loop completes, the full chronological list is attached to the final choice object:

response.choices[0].intermediate_messages = intermediate_messages

The Request Loop in aisuite/client.py

The workflow follows a predictable loop that runs until the model returns a final answer without requesting a tool call.

1. Initial request – The user’s message is sent to the model.
2. Model response – If the model decides to invoke a tool, the raw Response is stored in intermediate_responses and its Message is added to intermediate_messages.
3. Tool execution – The tool runs and produces its own Message objects (tool_messages), which are appended to intermediate_messages.
4. Loop – Steps 2–3 repeat until the model returns a final answer without a tool call.
5. Final packaging – The final Response is returned with intermediate_responses (all responses except the final one) and intermediate_messages (the full message history).

A simplified version of the loop in aisuite/client.py looks like this:

# Inside aisuite/client.py (simplified)

intermediate_responses = []
intermediate_messages = []

while not done:
    response = await model_api_call(...)
    intermediate_responses.append(response)
    intermediate_messages.append(response.choices[0].message)

    if response.choices[0].message.tool_calls:
        tool_messages = await run_tool_calls(...)
        intermediate_messages.extend(tool_messages)
    else:
        done = True

# Expose everything but the final response

response.intermediate_responses = intermediate_responses[:-1]
response.choices[0].intermediate_messages = intermediate_messages
return response

How Agents Access Intermediate Data

Downstream agents and utilities in the aisuite/agents/ directory expose the captured collections for logging, serialization, and pipeline execution.

Serializing Messages in aisuite/agents/utils.py

The helper utilities read intermediate_messages from the choice object to reconstruct the conversation for UI rendering or logging. In aisuite/agents/utils.py, the code safely retrieves the attribute using getattr:

# aisuite/agents/utils.py

def serialize_choice(choice):
    intermediate = getattr(choice, "intermediate_messages", None)
    if intermediate:
        return {"messages": messages_to_dicts(intermediate)}
    return {}

Propagating Responses in aisuite/agents/runner.py

The agent runner passes intermediate responses down the execution pipeline by unpacking the list. In aisuite/agents/runner.py, this is done with the following expression:

# aisuite/agents/runner.py

*getattr(response, "intermediate_responses", [])

Validation in Unit and Integration Tests

The behavior is verified in tests/client/test_client.py, where assertions confirm the exact counts of intermediate data:

# tests/client/test_client.py

def test_intermediate_handling():
    response = await client.chat(...)

    # One intermediate response before the final answer

    assert len(response.intermediate_responses) == 1

    # Three intermediate messages (user -> tool -> model)

    assert len(response.choices[0].intermediate_messages) == 3

End-to-end coverage across the MCP transport layer is maintained in tests/mcp/test_e2e.py.

Summary

• aisuite handles intermediate messages during tool-calling conversations through two parallel lists: intermediate_responses and intermediate_messages.
• The aisuite/client.py loop appends every model reply and tool output, then trims the responses with [:-1] and attaches the full message history to response.choices[0].
• Agents access this data via aisuite/agents/utils.py for serialization and aisuite/agents/runner.py for pipeline propagation.
• End-to-end validation lives in tests/client/test_client.py and tests/mcp/test_e2e.py.

Frequently Asked Questions

What is the difference between intermediate_responses and intermediate_messages?

intermediate_responses contains raw Response objects from the model after each tool call, while intermediate_messages contains Message objects representing every utterance in the conversation, including user prompts, tool outputs, and model replies. The responses are trimmed to exclude the final answer, whereas the messages preserve the complete chronological history.

Why does aisuite exclude the final response from intermediate_responses?

The framework slices the list with intermediate_responses[:-1] in aisuite/client.py because the final Response object is already the primary return value. Excluding it from the intermediate collection avoids duplication and keeps the metadata list focused on the prior tool-calling turns.

How can agents access intermediate_messages after a client call?

After the request loop finishes, the full message history is stored on response.choices[0].intermediate_messages. Agents in aisuite/agents/utils.py access this field using getattr(choice, "intermediate_messages", None), allowing safe serialization even when the attribute is absent.

Which test files validate intermediate message handling?

Unit-level assertions for counts and exposure reside in tests/client/test_client.py, which verifies that intermediate_responses and intermediate_messages contain the expected number of items. End-to-end coverage ensuring that intermediate messages survive across the MCP transport layer is found in tests/mcp/test_e2e.py.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}