# How to Handle WebSocket Connections for Real-Time Streaming Chat in AnythingLLM

> Master WebSocket connections in AnythingLLM for real-time chat. Learn how to implement low-latency agent responses and immediate session termination for seamless streaming.

- Repository: [Mintplex Labs/anything-llm](https://github.com/Mintplex-Labs/anything-llm)
- Tags: how-to-guide
- Published: 2026-03-07

---

**AnythingLLM implements WebSocket connections for real-time streaming chat through a bidirectional bridge at `/api/agent-invocation/:uuid`, enabling low-latency agent responses, mid-conversation user feedback, and immediate session termination via bail commands while maintaining compatibility with the standard HTTP streaming pipeline.**

AnythingLLM, the open-source AI workspace by Mintplex-Labs, supports two transport modes for delivering LLM responses: traditional HTTP Server-Sent Events and WebSocket connections for real-time streaming chat. While HTTP streaming handles standard requests through `POST /api/multiplex`, the WebSocket implementation provides bidirectional communication essential for interactive agent workflows that require user intervention during generation.

## WebSocket vs HTTP Streaming Architecture

AnythingLLM maintains a unified message pipeline that accepts data from two distinct transport layers:

- **HTTP Streaming**: Uses Server-Sent Events via `POST /api/multiplex`. The `handleChat` utility in [`frontend/src/utils/chat/index.js`](https://github.com/Mintplex-Labs/anything-llm/blob/main/frontend/src/utils/chat/index.js) parses sequences of `textResponseChunk` and `finalizeResponseStream` messages.
- **WebSocket Streaming**: Upgrades to `ws://.../api/agent-invocation/:uuid`. The UI opens a persistent socket, receives JSON messages from the WebSocket plugin attached to the agent, and feeds them into the same chat-history pipeline used by HTTP streaming.

## End-to-End WebSocket Connection Flow

### Initiating the WebSocket Session

When a user requests an agent-based chat, the `Workspace.multiplexStream` method returns a special initialization message. In [`frontend/src/utils/chat/index.js`](https://github.com/Mintplex-Labs/anything-llm/blob/main/frontend/src/utils/chat/index.js) (lines 40-42), the `handleChat` function detects when `type === "agentInitWebsocketConnection"` and extracts the UUID:

```js
// inside handleChat → type === "agentInitWebsocketConnection"
setWebsocket(chatResult.websocketUUID);

```

This UUID uniquely identifies the agent invocation session.

### Client-Side Socket Creation

When the `socketId` state becomes defined, the `ChatContainer` component in [`frontend/src/components/WorkspaceChat/ChatContainer/index.jsx`](https://github.com/Mintplex-Labs/anything-llm/blob/main/frontend/src/components/WorkspaceChat/ChatContainer/index.jsx) (lines 55-58) constructs the WebSocket:

```js
const socket = new WebSocket(
  `${websocketURI()}/api/agent-invocation/${socketId}`
);

```

The `websocketURI` helper, defined in [`frontend/src/utils/chat/agent.js`](https://github.com/Mintplex-Labs/anything-llm/blob/main/frontend/src/utils/chat/agent.js) (lines 19-23), automatically selects `ws:` or `wss:` based on the current page protocol to ensure secure connections in production environments.

### Server-Side Upgrade and Agent Binding

The server endpoint in [`server/endpoints/agentWebsocket.js`](https://github.com/Mintplex-Labs/anything-llm/blob/main/server/endpoints/agentWebsocket.js) (lines 20-53) handles the upgrade:

```js
// server/endpoints/agentWebsocket.js
app.ws('/agent-invocation/:uuid', async (socket, request) => {
  const handler = await new AgentHandler({
    uuid: String(request.params.uuid),
  }).init();

  if (!handler.invocation) { socket.close(); return; }

  // attach WS plugin so the agent can talk through the socket
  await handler.createAIbitat({ socket });
  await handler.startAgentCluster();
});

```

At lines 51-52, the `AgentHandler` creates an Aibitat instance with the raw socket attached, enabling the agent to communicate through the WebSocket bridge.

### The WebSocket Plugin Lifecycle

The WebSocket plugin in [`server/utils/agents/aibitat/plugins/websocket.js`](https://github.com/Mintplex-Labs/anything-llm/blob/main/server/utils/agents/aibitat/plugins/websocket.js) registers several critical callbacks:

- **onMessage**: Forwards every agent chat message to the client via `socket.send(JSON.stringify(message))`
- **onError / onTerminate**: Sends `wssFailure` notifications or closes the socket
- **onInterrupt**: Implements the feedback loop by invoking `socket.askForFeedback`
- **Introspection**: Optional "thought-bubble" status updates via `statusResponse`

The plugin also defines bail commands (`exit`, `/stop`, etc.) in `WEBSOCKET_BAIL_COMMANDS` that immediately abort the session when received from the client.

### Handling Real-Time Messages

On the frontend, every incoming message event routes through `handleSocketResponse` in [`frontend/src/utils/chat/agent.js`](https://github.com/Mintplex-Labs/anything-llm/blob/main/frontend/src/utils/chat/agent.js) (lines 25-54, 81-124):

```js
export default function handleSocketResponse(socket, event, setChatHistory) {
  const data = safeJsonParse(event.data, null);
  if (!data) return;

  // stream‑event tells us the provider supports agent‑side streaming
  if (data.type === 'reportStreamEvent') {
    socket.supportsAgentStreaming = true;
    // ... update chat history with status, chunks, etc.
  }

  // file download example
  if (data.type === 'fileDownload') {
    saveAs(data.content.b64Content, data.content.filename ?? 'unknown.txt');
    return;
  }

  // generic fallback – add a new assistant message
  setChatHistory(prev => [
    ...prev.filter(m => !!m.content),
    { uuid: v4(), type: data.type, content: data.content, role: 'assistant' }
  ]);
}

```

This function normalizes WebSocket payloads into the same chat-history model used by HTTP streaming, handling special cases like `textResponseChunk`, file downloads, and failure notifications.

### Session Termination

When the agent finishes execution, the plugin calls `socket.close()`. The UI's `close` listener in [`frontend/src/components/WorkspaceChat/ChatContainer/index.jsx`](https://github.com/Mintplex-Labs/anything-llm/blob/main/frontend/src/components/WorkspaceChat/ChatContainer/index.jsx) (lines 78-95) injects a final "Agent session complete" status message and resets the socket state, ensuring clean resource cleanup.

## Why WebSocket Connections for Real-Time Streaming Chat?

The WebSocket implementation in AnythingLLM provides three distinct advantages over HTTP streaming:

- **Bidirectional, low-latency communication**: Allows the agent to request mid-conversation feedback via `awaitingFeedback` states without round-trip HTTP requests, essential for interactive tool use and human-in-the-loop workflows.
- **Streaming of partial results**: The UI displays `textResponseChunk` messages immediately upon arrival, creating a smooth "typing" experience while the agent generates responses.
- **Graceful bail-out**: Users can type `/stop` or other `WEBSOCKET_BAIL_COMMANDS` to abort instantly; the server reacts by calling `agentHandler.aibitat.abort()` and closing the socket, preserving system resources.

## Critical Source Files

| Area | File | Purpose |
|------|------|---------|
| **WebSocket endpoint** | [`server/endpoints/agentWebsocket.js`](https://github.com/Mintplex-Labs/anything-llm/blob/main/server/endpoints/agentWebsocket.js) | Express-WS route that creates the `AgentHandler`, injects the raw socket, and starts the agent cluster. |
| **WebSocket plugin** | [`server/utils/agents/aibitat/plugins/websocket.js`](https://github.com/Mintplex-Labs/anything-llm/blob/main/server/utils/agents/aibitat/plugins/websocket.js) | Hooks into the Aibitat agent to forward messages, handle errors, manage introspection, and process bail commands. |
| **Frontend WebSocket helper** | [`frontend/src/utils/chat/agent.js`](https://github.com/Mintplex-Labs/anything-llm/blob/main/frontend/src/utils/chat/agent.js) | `websocketURI` builder and `handleSocketResponse` that parses inbound JSON and updates React chat state. |
| **Chat UI integration** | [`frontend/src/components/WorkspaceChat/ChatContainer/index.jsx`](https://github.com/Mintplex-Labs/anything-llm/blob/main/frontend/src/components/WorkspaceChat/ChatContainer/index.jsx) | React component that detects `socketId`, opens the socket, wires event listeners, and handles session completion. |
| **Chat dispatcher** | [`frontend/src/utils/chat/index.js`](https://github.com/Mintplex-Labs/anything-llm/blob/main/frontend/src/utils/chat/index.js) | `handleChat` processes normal streaming and the `agentInitWebsocketConnection` message that triggers WS flow. |

## Summary

- AnythingLLM uses **WebSocket connections for real-time streaming chat** to enable bidirectional communication between the browser and Aibitat agents.
- The connection initiates when the server returns an `agentInitWebsocketConnection` message containing a UUID, which the frontend uses to open `ws://.../api/agent-invocation/:uuid`.
- The **WebSocket plugin** ([`server/utils/agents/aibitat/plugins/websocket.js`](https://github.com/Mintplex-Labs/anything-llm/blob/main/server/utils/agents/aibitat/plugins/websocket.js)) manages the lifecycle, forwarding agent messages, handling interrupts for feedback, and processing bail commands like `/stop`.
- Frontend message handling is unified through `handleSocketResponse` in [`frontend/src/utils/chat/agent.js`](https://github.com/Mintplex-Labs/anything-llm/blob/main/frontend/src/utils/chat/agent.js), which normalizes WebSocket payloads into the same chat-history model used by HTTP streaming.
- This architecture supports low-latency partial response streaming, mid-conversation user feedback, and immediate session abortion while maintaining a consistent UI state across transport modes.

## Frequently Asked Questions

### What is the difference between HTTP streaming and WebSocket streaming in AnythingLLM?

HTTP streaming uses Server-Sent Events via `POST /api/multiplex` to deliver `textResponseChunk` messages in a unidirectional stream. WebSocket streaming upgrades the connection to `ws://.../api/agent-invocation/:uuid`, enabling bidirectional communication that allows the agent to request mid-conversation feedback and process abort commands instantly.

### How does the WebSocket connection handle user interruptions during agent execution?

The WebSocket plugin registers an `onInterrupt` callback that pauses the agent and invokes `socket.askForFeedback`, sending an `awaitingFeedback` status to the client. The UI can then submit user input back through the same socket, which the plugin forwards via `aibitat.continue(feedback)`, resuming the agent workflow without requiring a new HTTP request.

### What are bail commands in AnythingLLM's WebSocket implementation?

Bail commands are specific text inputs defined in `WEBSOCKET_BAIL_COMMANDS` (such as `exit`, `/stop`, or `quit`) that users can type to immediately abort an active agent session. When received, the WebSocket plugin calls `agentHandler.aibitat.abort()` and closes the socket, terminating the session and freeing server resources.

### How does the frontend distinguish between WebSocket and HTTP streaming modes?

The frontend relies on the message type returned by the initial chat request. When `handleChat` in [`frontend/src/utils/chat/index.js`](https://github.com/Mintplex-Labs/anything-llm/blob/main/frontend/src/utils/chat/index.js) encounters `type === "agentInitWebsocketConnection"`, it extracts the `websocketUUID` and triggers the WebSocket flow. If the response contains standard `textResponseChunk` messages instead, the UI processes them as HTTP streaming through the same `handleChat` pipeline.