# How Agent Zero Handles User Input and Responses: A Deep Dive into the Message Pipeline

> Discover how Agent Zero processes user input and responses through its message pipeline normalizing requests, managing history, streaming LLM replies, and executing tool calls.

- Repository: [Agent Zero/agent-zero](https://github.com/agent0ai/agent-zero)
- Tags: deep-dive
- Published: 2026-02-23

---

**Agent Zero normalizes all inbound requests into a `UserMessage` object, stores them in conversation history, streams LLM responses through extensible callbacks, and executes any embedded tool calls before returning the final output.**

Agent Zero is an open-source agentic framework that orchestrates large language model interactions with tool execution capabilities. Understanding how Agent Zero handles user input and responses reveals a carefully architected pipeline that ensures consistent message normalization, extensible stream processing, and secure command execution. The framework transforms every request into a canonical data structure and processes it through four distinct stages before returning a rendered response.

## Reception and Normalization with UserMessage

All external input—whether from HTTP POST requests, WebSocket connections, or internal API calls—converges on a single abstraction. In [`python/api/message.py`](https://github.com/agent0ai/agent-zero/blob/main/python/api/message.py), the endpoint constructs a `UserMessage` instance and passes it to the agent's communication loop.

```python

# From python/api/message.py (line 71)

return context.communicate(UserMessage(message, attachment_paths)), context

```

The `communicate` method on `AgentContext` receives this object and initiates the processing pipeline. This design ensures that file attachments, system messages, and plain text are handled uniformly regardless of transport protocol.

### The UserMessage Data Class

The canonical representation of user input is defined in [`agent.py`](https://github.com/agent0ai/agent-zero/blob/main/agent.py) as a lightweight dataclass:

```python

# From agent.py (lines 319-324)

@dataclass
class UserMessage:
    message: str
    attachments: list[str] = field(default_factory=list[str])
    system_message: list[str] = field(default_factory=list[str])

```

This structure accommodates optional file paths alongside the primary message text, enabling multimodal workflows where users attach documents or images for the agent to analyze.

## History Management and Prompt Construction

Once normalized, the `UserMessage` is persisted to the agent's conversation history via `hist_add_user_message`. During the next iteration of `Agent.monologue`, the framework assembles a prompt composed of system instructions, recent conversation turns, and any context injected by registered extensions.

This history-aware architecture allows the LLM to reference earlier parts of the conversation when formulating responses. The assembled prompt is then passed to `call_chat_model` to initiate the inference request.

## LLM Invocation and Streaming Response Processing

Agent Zero implements a dual-channel streaming architecture where the LLM emits both reasoning traces and the final user-visible response simultaneously. The `monologue` method manages this through an async callback system:

```python

# From agent.py (lines 35-43)

async def stream_callback(chunk: str, full: str):
    await self.handle_intervention()
    if chunk == full:
        printer.print("Response: ")
    stream_data = {"chunk": chunk, "full": full}
    await self.call_extensions("response_stream_chunk", stream_data=stream_data)
    if stream_data.get("chunk"):
        printer.stream(stream_data["chunk"])
    await self.handle_response_stream(stream_data["full"])

```

The callback invokes `call_extensions("response_stream_chunk", ...)` at every chunk, allowing middleware to filter, redact, or enrich content before it reaches the UI. This extensibility point is critical for implementing security policies, logging, or real-time formatting without modifying core engine code.

## Tool Extraction and Execution

When the LLM finishes generating, Agent Zero inspects the response text for embedded JSON tool requests. The `process_tools` method in [`agent.py`](https://github.com/agent0ai/agent-zero/blob/main/agent.py) handles this extraction and dispatch:

```python

# From agent.py (lines 55-72)

tool_request = extract_tools.json_parse_dirty(msg)
if tool_request is not None:
    raw_tool_name = tool_request.get("tool_name", tool_request.get("tool",""))
    tool_args = tool_request.get("tool_args", tool_request.get("args", {}))
    # ... tool resolution logic ...

    tool = self.get_tool(name=tool_name, method=tool_method, args=tool_args, ...)
    if tool:
        response = await tool.execute(**tool_args)

```

After execution, results are added back to the conversation history via `hist_add_tool_result`, allowing the LLM to incorporate tool outputs into subsequent reasoning or final answers.

### The Terminal Input Tool Example

When the LLM requests user terminal access via the `"input"` tool, the request flows through [`python/tools/input.py`](https://github.com/agent0ai/agent-zero/blob/main/python/tools/input.py):

```python

# From python/tools/input.py (lines 8-20)

async def execute(self, keyboard="", **kwargs):
    keyboard = keyboard.rstrip()
    session = int(self.args.get("session", 0))
    args = {"runtime": "terminal", "code": keyboard, "session": session, "allow_running": True}
    cet = CodeExecution(self.agent, "code_execution_tool", "", args, self.message, self.loop_data)
    return await cet.execute(**args)

```

This tool forwards the keyboard input to the `CodeExecution` class, which runs commands in a sandboxed terminal session. The output is captured and returned to the agent, creating a secure loop where the LLM can execute shell commands without exposing the host system to uncontrolled access.

## Summary

- **Unified Input Abstraction**: All requests become `UserMessage` objects in [`agent.py`](https://github.com/agent0ai/agent-zero/blob/main/agent.py), standardizing handling across HTTP, WebSocket, and internal APIs.
- **Persistent History**: The `hist_add_user_message` and `hist_add_tool_result` functions maintain conversation context for multi-turn reasoning.
- **Extensible Streaming**: The `stream_callback` in `Agent.monologue` enables real-time modifications through the `response_stream_chunk` extension point.
- **Secure Tool Execution**: `process_tools` parses JSON payloads and dispatches to sandboxed tools like `CodeExecution`, with results reintegrated into the conversation history.

## Frequently Asked Questions

### What is the UserMessage class in Agent Zero?

The `UserMessage` class is a dataclass defined in [`agent.py`](https://github.com/agent0ai/agent-zero/blob/main/agent.py) (lines 319-324) that serves as the canonical container for all user input. It stores the message string, a list of attachment paths, and optional system messages, ensuring consistent data structures throughout the pipeline regardless of whether input arrives via HTTP API or WebSocket.

### How does Agent Zero stream LLM responses to the user interface?

Agent Zero uses an async `stream_callback` function within the `monologue` method that receives text chunks as the LLM generates them. Each chunk is wrapped in a `stream_data` dictionary and passed through the `response_stream_chunk` extension hook, allowing middleware to modify content before `printer.stream()` renders it to the UI.

### What happens when the LLM requests a tool execution?

When the LLM emits a JSON tool request, the `process_tools` method (lines 55-72 in [`agent.py`](https://github.com/agent0ai/agent-zero/blob/main/agent.py)) parses the payload using `extract_tools.json_parse_dirty`, resolves the appropriate tool instance via `get_tool`, and invokes `await tool.execute()`. The result is stored in history via `hist_add_tool_result`, and the conversation continues with the tool output available as context.

### How does Agent Zero handle file attachments with user messages?

File paths are passed as the `attachments` parameter when constructing a `UserMessage` in [`python/api/message.py`](https://github.com/agent0ai/agent-zero/blob/main/python/api/message.py). These paths are stored alongside the text content in the dataclass and can be accessed by tools or extensions during prompt construction, enabling the agent to read and analyze uploaded documents.