How to Handle Multi-Turn Conversations with Tool Calls and Results in Needle

Needle enables multi-turn conversations with tool calls by maintaining a conversation history that cycles through user prompts, model-generated tool invocations, tool execution results, and follow-up model responses until a final answer is produced.

The needle library provides a conversational agent architecture where the language model can invoke Python tools during a chat, receive their results, and continue the dialogue based on those outcomes. This creates rich, multi-turn interactions where the model can fetch data, run code, or query external services and seamlessly incorporate returned information into subsequent turns.

The Multi-Turn Tool Call Loop

Needle implements a five-step cycle for handling multi-turn conversations with tool calls:

1. User Prompt → Agent — The user submits a message. The agent parses the request and decides whether a tool should be invoked.

2. Tool Invocation — If needed, the agent generates a function-call specification (name, arguments) and sends it to the runtime.

3. Tool Execution → Result — The runtime executes the requested tool and returns the result, typically as JSON.

4. Result ↪ Conversation State — The result is stored in the conversation context for reference in the next turn.

5. Follow-up Prompt → Model — The model receives the user message, tool-call metadata, and tool result to produce a response incorporating the fresh data or decide on additional tool calls.

This loop repeats arbitrarily, enabling progressive refinement of answers, information gathering, or chains of dependent operations.

Core Architectural Components

needle/agent/tools.py — The Toolbox

Defines callable functions the model can invoke, including fetch, run, and save. Each function is a regular Python callable returning serializable data.

needle/agent/fetch.py — HTTP Tool Implementation

Implements the fetch tool for retrieving remote resources. Handles HTTP requests, caching, and error handling.

needle/cli.py — Interactive Session Driver

Entry point that sets up the agent, parses command-line arguments, and starts the interactive loop. Wires the model with the toolbox and maintains conversation history.

needle/model/run.py — Inference Engine

Core inference logic that runs the language model, injects tool-call metadata, and processes tool results for the next turn.

needle/model/architecture.py — Model Definition

Describes the model's transformer architecture and its interface with the tool-calling subsystem.

How Multi-Turn State Is Managed Internally

Conversation Store — Each turn's user message, model output, and tool results append to a conversation_history list passed back to the model on every inference step.

Tool-Call Detection — After model generation, needle/model/run.py scans output for JSON blocks matching the function-call schema (name + arguments).

Result Serialization — Tool functions return plain Python objects. The runtime serializes these to JSON before injection into the next model prompt.

Loop Termination — The cycle continues until the model produces a final answer without a pending tool call, or the user explicitly ends the session.

Practical Code Examples

Example 1: Fetch and Summarize Web Content

from needle import NeedleAgent

agent = NeedleAgent()
agent.ask("Please summarize the latest news about AI from https://example.com/ai-news.")

• Turn 1: Model calls fetch(url="https://example.com/ai-news")
• Turn 2: fetch returns raw HTML; model receives payload and generates summary

Example 2: Chained Tool Operations

agent.ask(
    "Download the CSV at https://data.org/stats.csv, compute the average of column 'score', "
    "and store the result in a file called avg.txt."
)

1. First call: fetch(url="https://data.org/stats.csv") → CSV data
2. Second call: run(code="import pandas as pd; df=pd.read_csv(...); avg=df['score'].mean()") → numeric average
3. Third call: save(path="avg.txt", content=str(avg)) → persists result

The agent automatically tracks dependencies, feeding each tool's output into subsequent prompts.

Example 3: Interactive Search with User Clarification

agent.ask("Find the Python library that provides a fast JSON parser.")

• Turn 1: Model invokes fetch on search API, receives candidate list
• Turn 2: Model asks user: "Do you prefer a pure-Python implementation or a compiled extension?"
• Turn 3: Based on clarification, model calls fetch with refined query, then responds with chosen library (e.g., orjson)

Summary

• Multi-turn conversations with tool calls in Needle follow a structured loop: prompt → tool invocation → execution → result injection → follow-up generation
• needle/model/run.py handles tool-call detection and result injection into the conversation state
• needle/agent/tools.py and needle/agent/fetch.py provide the callable utilities for external actions
• Conversation history persistence enables arbitrary-length reasoning chains with dependent operations
• Automatic serialization of tool results ensures seamless model consumption of structured data

Frequently Asked Questions

How does Needle detect when to make a tool call?

Needle scans model output in needle/model/run.py for JSON blocks matching the function-call schema with name and arguments fields. When detected, the runtime executes the specified tool rather than returning the output directly to the user.

Can tool calls depend on previous tool results?

Yes. Needle's conversation state includes all prior tool results, which the model can reference in subsequent turns. This enables chains where one tool's output becomes input for the next tool call or reasoning step.

What happens if a tool execution fails?

Tool functions in needle/agent/tools.py implement error handling and return serializable error information. The model receives this error data and can either retry with modified parameters, request user clarification, or explain the failure in its response.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}