How to Handle WebSocket Connections for Real-Time Streaming Chat in AnythingLLM

AnythingLLM implements WebSocket connections for real-time streaming chat through a bidirectional bridge at /api/agent-invocation/:uuid, enabling low-latency agent responses, mid-conversation user feedback, and immediate session termination via bail commands while maintaining compatibility with the standard HTTP streaming pipeline.

AnythingLLM, the open-source AI workspace by Mintplex-Labs, supports two transport modes for delivering LLM responses: traditional HTTP Server-Sent Events and WebSocket connections for real-time streaming chat. While HTTP streaming handles standard requests through POST /api/multiplex, the WebSocket implementation provides bidirectional communication essential for interactive agent workflows that require user intervention during generation.

WebSocket vs HTTP Streaming Architecture

AnythingLLM maintains a unified message pipeline that accepts data from two distinct transport layers:

• HTTP Streaming: Uses Server-Sent Events via POST /api/multiplex. The handleChat utility in frontend/src/utils/chat/index.js parses sequences of textResponseChunk and finalizeResponseStream messages.
• WebSocket Streaming: Upgrades to ws://.../api/agent-invocation/:uuid. The UI opens a persistent socket, receives JSON messages from the WebSocket plugin attached to the agent, and feeds them into the same chat-history pipeline used by HTTP streaming.

End-to-End WebSocket Connection Flow

Initiating the WebSocket Session

When a user requests an agent-based chat, the Workspace.multiplexStream method returns a special initialization message. In frontend/src/utils/chat/index.js (lines 40-42), the handleChat function detects when type === "agentInitWebsocketConnection" and extracts the UUID:

// inside handleChat → type === "agentInitWebsocketConnection"
setWebsocket(chatResult.websocketUUID);

This UUID uniquely identifies the agent invocation session.

Client-Side Socket Creation

When the socketId state becomes defined, the ChatContainer component in frontend/src/components/WorkspaceChat/ChatContainer/index.jsx (lines 55-58) constructs the WebSocket:

const socket = new WebSocket(
  `${websocketURI()}/api/agent-invocation/${socketId}`
);

The websocketURI helper, defined in frontend/src/utils/chat/agent.js (lines 19-23), automatically selects ws: or wss: based on the current page protocol to ensure secure connections in production environments.

Server-Side Upgrade and Agent Binding

The server endpoint in server/endpoints/agentWebsocket.js (lines 20-53) handles the upgrade:

// server/endpoints/agentWebsocket.js
app.ws('/agent-invocation/:uuid', async (socket, request) => {
  const handler = await new AgentHandler({
    uuid: String(request.params.uuid),
  }).init();

  if (!handler.invocation) { socket.close(); return; }

  // attach WS plugin so the agent can talk through the socket
  await handler.createAIbitat({ socket });
  await handler.startAgentCluster();
});

At lines 51-52, the AgentHandler creates an Aibitat instance with the raw socket attached, enabling the agent to communicate through the WebSocket bridge.

The WebSocket Plugin Lifecycle

The WebSocket plugin in server/utils/agents/aibitat/plugins/websocket.js registers several critical callbacks:

• onMessage: Forwards every agent chat message to the client via socket.send(JSON.stringify(message))
• onError / onTerminate: Sends wssFailure notifications or closes the socket
• onInterrupt: Implements the feedback loop by invoking socket.askForFeedback
• Introspection: Optional "thought-bubble" status updates via statusResponse

The plugin also defines bail commands (exit, /stop, etc.) in WEBSOCKET_BAIL_COMMANDS that immediately abort the session when received from the client.

Handling Real-Time Messages

On the frontend, every incoming message event routes through handleSocketResponse in frontend/src/utils/chat/agent.js (lines 25-54, 81-124):

export default function handleSocketResponse(socket, event, setChatHistory) {
  const data = safeJsonParse(event.data, null);
  if (!data) return;

  // stream‑event tells us the provider supports agent‑side streaming
  if (data.type === 'reportStreamEvent') {
    socket.supportsAgentStreaming = true;
    // ... update chat history with status, chunks, etc.
  }

  // file download example
  if (data.type === 'fileDownload') {
    saveAs(data.content.b64Content, data.content.filename ?? 'unknown.txt');
    return;
  }

  // generic fallback – add a new assistant message
  setChatHistory(prev => [
    ...prev.filter(m => !!m.content),
    { uuid: v4(), type: data.type, content: data.content, role: 'assistant' }
  ]);
}

This function normalizes WebSocket payloads into the same chat-history model used by HTTP streaming, handling special cases like textResponseChunk, file downloads, and failure notifications.

Session Termination

When the agent finishes execution, the plugin calls socket.close(). The UI's close listener in frontend/src/components/WorkspaceChat/ChatContainer/index.jsx (lines 78-95) injects a final "Agent session complete" status message and resets the socket state, ensuring clean resource cleanup.

Why WebSocket Connections for Real-Time Streaming Chat?

The WebSocket implementation in AnythingLLM provides three distinct advantages over HTTP streaming:

• Bidirectional, low-latency communication: Allows the agent to request mid-conversation feedback via awaitingFeedback states without round-trip HTTP requests, essential for interactive tool use and human-in-the-loop workflows.
• Streaming of partial results: The UI displays textResponseChunk messages immediately upon arrival, creating a smooth "typing" experience while the agent generates responses.
• Graceful bail-out: Users can type /stop or other WEBSOCKET_BAIL_COMMANDS to abort instantly; the server reacts by calling agentHandler.aibitat.abort() and closing the socket, preserving system resources.

Critical Source Files

Area	File	Purpose
WebSocket endpoint	server/endpoints/agentWebsocket.js	Express-WS route that creates the AgentHandler, injects the raw socket, and starts the agent cluster.
WebSocket plugin	server/utils/agents/aibitat/plugins/websocket.js	Hooks into the Aibitat agent to forward messages, handle errors, manage introspection, and process bail commands.
Frontend WebSocket helper	frontend/src/utils/chat/agent.js	websocketURI builder and handleSocketResponse that parses inbound JSON and updates React chat state.
Chat UI integration	frontend/src/components/WorkspaceChat/ChatContainer/index.jsx	React component that detects socketId, opens the socket, wires event listeners, and handles session completion.
Chat dispatcher	frontend/src/utils/chat/index.js	handleChat processes normal streaming and the agentInitWebsocketConnection message that triggers WS flow.

Summary

• AnythingLLM uses WebSocket connections for real-time streaming chat to enable bidirectional communication between the browser and Aibitat agents.
• The connection initiates when the server returns an agentInitWebsocketConnection message containing a UUID, which the frontend uses to open ws://.../api/agent-invocation/:uuid.
• The WebSocket plugin (server/utils/agents/aibitat/plugins/websocket.js) manages the lifecycle, forwarding agent messages, handling interrupts for feedback, and processing bail commands like /stop.
• Frontend message handling is unified through handleSocketResponse in frontend/src/utils/chat/agent.js, which normalizes WebSocket payloads into the same chat-history model used by HTTP streaming.
• This architecture supports low-latency partial response streaming, mid-conversation user feedback, and immediate session abortion while maintaining a consistent UI state across transport modes.

Frequently Asked Questions

What is the difference between HTTP streaming and WebSocket streaming in AnythingLLM?

HTTP streaming uses Server-Sent Events via POST /api/multiplex to deliver textResponseChunk messages in a unidirectional stream. WebSocket streaming upgrades the connection to ws://.../api/agent-invocation/:uuid, enabling bidirectional communication that allows the agent to request mid-conversation feedback and process abort commands instantly.

How does the WebSocket connection handle user interruptions during agent execution?

The WebSocket plugin registers an onInterrupt callback that pauses the agent and invokes socket.askForFeedback, sending an awaitingFeedback status to the client. The UI can then submit user input back through the same socket, which the plugin forwards via aibitat.continue(feedback), resuming the agent workflow without requiring a new HTTP request.

What are bail commands in AnythingLLM's WebSocket implementation?

Bail commands are specific text inputs defined in WEBSOCKET_BAIL_COMMANDS (such as exit, /stop, or quit) that users can type to immediately abort an active agent session. When received, the WebSocket plugin calls agentHandler.aibitat.abort() and closes the socket, terminating the session and freeing server resources.

How does the frontend distinguish between WebSocket and HTTP streaming modes?

The frontend relies on the message type returned by the initial chat request. When handleChat in frontend/src/utils/chat/index.js encounters type === "agentInitWebsocketConnection", it extracts the websocketUUID and triggers the WebSocket flow. If the response contains standard textResponseChunk messages instead, the UI processes them as HTTP streaming through the same handleChat pipeline.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}