How to Integrate MCP (Model Context Protocol) Servers with AnythingLLM: A Complete Guide

AnythingLLM treats MCP servers as dynamic plugins that expose external tools as agent functions through the MCPHypervisor and MCPCompatibilityLayer classes, enabling seamless connections via stdio, HTTP, or SSE transports.

The Model Context Protocol (MCP) is an open standard that allows AI systems to connect to external data sources and tools. AnythingLLM implements native MCP support through a sophisticated hypervisor architecture that converts MCP server capabilities into callable functions for the Aibitat agent runtime. This integration enables your workspace agents to interact with databases, file systems, APIs, and specialized tools using a standardized protocol.

Understanding the MCP Integration Architecture in AnythingLLM

The integration relies on three core components working together to bridge MCP servers with the AnythingLLM agent system.

The MCPHypervisor Class

Located in server/utils/MCP/hypervisor/index.js, the MCPHypervisor manages the lifecycle of MCP server processes. Its constructor automatically creates the configuration file at storage/plugins/anythingllm_mcp_servers.json if it does not exist (lines 67-78). The hypervisor validates server definitions, builds appropriate transports (stdio, HTTP, or SSE), and spawns processes through the private #startMCPServer method (lines 89-105).

The MCPCompatibilityLayer Class

The MCPCompatibilityLayer in server/utils/MCP/index.js serves as the translation layer between MCP servers and the Aibitat agent runtime. It queries running servers for available tools via listTools(), then constructs plugin objects that register functions with aibitat.function and tags them with isMCPTool: true for special cooldown handling (lines 44-62). This layer also handles result serialization through returnMCPResult (lines 78-89).

Workspace Agent Integration

The WORKSPACE_AGENT.getDefinition method in server/utils/agents/defaults.js (lines 35-44) merges built-in skills, imported plugins, flow plugins, and MCP placeholders. Active servers contribute placeholder strings in the format @@mcp_<name>, which are dynamically replaced with full tool plugin definitions when the agent configuration is built.

Step-by-Step MCP Server Integration Process

Integrating an MCP server requires five distinct phases, from configuration definition to runtime execution.

Step 1: Define Your MCP Server Configuration

Create a JSON entry in storage/plugins/anythingllm_mcp_servers.json. The hypervisor reads this file to determine how to launch each server, supporting both stdio-based commands and HTTP/SSE endpoints.

{
  "mcpServers": {
    "my-docker-mcp": {
      "command": "docker",
      "args": ["run", "--rm", "-i", "my-mcp-image", "node", "server.js"],
      "env": {
        "PORT": "8080"
      }
    },
    "my-http-mcp": {
      "url": "http://localhost:5000",
      "type": "http",
      "headers": { "Authorization": "Bearer xyz" }
    }
  }
}

Step 2: Boot the Server via MCPHypervisor

Use the startMCPServer(name) method to validate the definition and initialize the transport. This method supports stdio processes, HTTP connections, and SSE streams, automatically handling environment variables and authentication headers defined in the configuration.

const MCP = require("../utils/MCP");

(async () => {
  const mcp = new MCP();
  await mcp.startMCPServer('my-docker-mcp');
  const active = await mcp.activeMCPServers();
  console.log('Active MCP placeholders:', active);
  // Output: ['@@mcp_my-docker-mcp']
})();

Step 3: Expose Tools as Agent Functions

Once running, convertServerToolsToPlugins(name) queries the server for its tool inventory and builds corresponding Aibitat function plugins. Each tool becomes available as a function named <server-name>-<tool-name> with the tool's JSON schema mapped to function parameters.

Step 4: Add MCP Placeholders to Workspace Agents

The workspace agent automatically incorporates MCP capabilities. When WORKSPACE_AGENT.getDefinition executes, it calls activeMCPServers() to retrieve current placeholders and merges them into the agent's function list alongside built-in skills and imported plugins.

const { WORKSPACE_AGENT } = require("./agents/defaults");

(async () => {
  const definition = await WORKSPACE_AGENT.getDefinition('openai', workspace, user);
  console.log(definition.functions.map(f => f.name));
  // Includes: "my-docker-mcp-listContainers", etc.
})();

Step 5: Invoke Tools from Chat

During conversation, the LLM can request function calls using the naming convention <server-name>-<tool-name>. The handler created by MCPCompatibilityLayer forwards these calls to the active MCP server via currentMcp.callTool and returns serialized results through returnMCPResult, making the external tool's output available to the conversation context.

Configuring MCP Server Definitions

AnythingLLM supports multiple transport mechanisms through the same configuration interface. For stdio transports, define the command and arguments array as shown in the Docker example above. For HTTP/SSE transports, specify the URL, type, and optional headers object. The hypervisor automatically selects the appropriate client implementation based on the type field, defaulting to stdio when unspecified.

Environment Configuration and Performance Tuning

MCP tools include a built-in cooldown mechanism to prevent infinite loops during agent execution. To disable this safety feature for trusted servers, set the environment variable MCP_NO_COOLDOWN to any value. This setting is processed in server/utils/helpers/updateENV.js (lines 1309-1310).

# In .env or runtime environment

MCP_NO_COOLDOWN=1

Summary

• MCPHypervisor in server/utils/MCP/hypervisor/index.js manages server lifecycle and transport initialization
• MCPCompatibilityLayer in server/utils/MCP/index.js converts MCP tools into Aibitat agent functions and handles result serialization
• Server definitions persist in storage/plugins/anythingllm_mcp_servers.json with support for stdio, HTTP, and SSE transports
• Workspace agents automatically integrate active MCP servers through @@mcp_<name> placeholders resolved by WORKSPACE_AGENT.getDefinition
• Tool invocation follows the naming pattern <server-name>-<tool-name> with automatic forwarding to the underlying MCP server
• Set MCP_NO_COOLDOWN=1 to disable the default cooldown protection for high-frequency tool usage

Frequently Asked Questions

What file format does AnythingLLM use to store MCP server definitions?

AnythingLLM stores MCP server configurations in storage/plugins/anythingllm_mcp_servers.json. This JSON file contains a mcpServers object where each key represents a server name and its value defines the command, arguments, environment variables, or HTTP endpoint details. The file is automatically created by the MCPHypervisor constructor if it does not exist when the server starts.

How does AnythingLLM handle different MCP transport types?

The MCPHypervisor class supports three transport mechanisms: stdio for local process spawning, HTTP for REST API connections, and SSE for server-sent event streams. The transport type is determined by the configuration—stdio is used when a command field is present, while HTTP or SSE is selected based on the type field in the server definition. The private #startMCPServer method builds the appropriate transport client accordingly.

Can I disable the automatic cooldown for MCP tools?

Yes. AnythingLLM implements a cooldown system to prevent MCP tools from being called in infinite loops. To disable this protection, set the environment variable MCP_NO_COOLDOWN to any truthy value in your .env file or runtime environment. This configuration is processed in server/utils/helpers/updateENV.js and removes the cooldown restriction from all MCP tool invocations.

How are MCP tools exposed to the LLM agent?

The MCPCompatibilityLayer.convertServerToolsToPlugins method queries active MCP servers using the listTools() protocol method, then registers each tool as an Aibitat function via aibitat.function. These functions are tagged with isMCPTool: true and named using the pattern <server-name>-<tool-name>. When the workspace agent definition is built, WORKSPACE_AGENT.getDefinition merges these functions alongside built-in skills and imported plugins, making them available for the LLM to invoke during chat sessions.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}