How the MCP Server Enables Agent Introspection in FreeLLMAPI

The FreeLLMAPI gateway implements a stateless MCP (Model Context Protocol) server at /mcp that allows coding agents to query runtime router state—such as available models, provider health, and routing strategies—through a JSON-RPC 2.0 interface without maintaining persistent sessions.

The Model Context Protocol is an open standard designed to let AI assistants interact with external systems. According to the FreeLLMAPI source code, the gateway exposes a lightweight JSON-RPC façade over its internal router state in server/src/routes/mcp.ts, enabling agents like Claude Code, Cursor, and Cline to introspect and even modify gateway behavior dynamically.

Stateless JSON-RPC Transport

Agents POST single JSON-RPC 2.0 messages to the /mcp endpoint. The implementation explicitly rejects batching per protocol version 2025-06-18 and returns plain JSON objects. The server is strictly stateless—GET or DELETE requests receive HTTP 405 errors explaining that no session storage is maintained.

Authentication mirrors the OpenAI-compatible /v1 endpoints. The authenticate() function validates Bearer tokens or x-api-key headers against the unified API key system. Invalid keys return JSON-RPC error -32001.

The Seven Introspection Tools

The TOOLS constant in server/src/routes/mcp.ts registers seven utilities, each defining a name, description, JSON-Schema inputSchema, and handler function.

Model Discovery with list_models

The list_models tool invokes buildModelListing() from server/src/services/model-listing.ts to compile a catalog of free models. The response includes context windows, tool support flags, and per-platform parameters, formatted as structured JSON.

Provider Health Monitoring with provider_health

This tool queries the SQLite database via getDb() to compute provider status counts, active rate-limit cool-downs, and usable model tallies. Agents use this to detect provider outages before routing requests.

Usage Analytics with usage_summary

Aggregating hourly statistics from the database, usage_summary accepts time ranges (24h, 7d, 30d) and returns request totals, token counts, success rates, and top-traffic models via the usageSummary() handler.

Routing Control with routing_info and set_routing_strategy

The routing_info tool exposes the active strategy and top-scored fallback models. Through set_routing_strategy, agents invoke setRoutingStrategy() to switch policies dynamically without restarting the gateway.

Performance Metrics with cache_stats and compression_stats

cache_stats retrieves hit/miss counters from server/src/services/cache.ts, while compression_stats pulls metrics from server/src/services/compression/stats.ts. These reveal quota savings from the compression engine and caching layer.

The Introspection Flow: Three-Step Protocol

The dispatchRpc() function orchestrates interactions through a standardized lifecycle:

1. Initialize: Agents call initialize to receive protocol capabilities and version 2025-06-18.
2. List Tools: Agents request tools/list to discover available utilities and their input schemas.
3. Call Tools: Agents invoke tools/call with a tool name and arguments, triggering the specific handler and returning a JSON-RPC result envelope containing tool-specific data.

POST /mcp
{
  "jsonrpc": "2.0",
  "id": 1,
  "method": "initialize",
  "params": {}
}

POST /mcp
{
  "jsonrpc": "2.0",
  "id": 2,
  "method": "tools/list",
  "params": {}
}

POST /mcp
{
  "jsonrpc": "2.0",
  "id": 3,
  "method": "tools/call",
  "params": {
    "name": "set_routing_strategy",
    "arguments": { "strategy": "smartest" }
  }
}

Stateless Architecture and Implementation

Registered in server/src/app.ts, the MCP router maintains zero session state. Each request is processed independently by dispatchRpc(), which routes to the appropriate handler based on the method field. This design eliminates connection overhead while providing full runtime visibility into routing decisions, provider status, and optimization metrics stored in the SQLite backend.

Summary

• Stateless JSON-RPC Endpoint: The /mcp route in server/src/routes/mcp.ts processes single-message JSON-RPC 2.0 requests, rejecting batching and returning 405 errors for non-POST methods.
• Unified Authentication: Uses the existing API key system from OpenAI-compatible endpoints, returning error code -32001 for invalid credentials via the authenticate() function.
• Seven Introspection Tools: Exposes list_models, provider_health, usage_summary, routing_info, set_routing_strategy, cache_stats, and compression_stats through the TOOLS registry.
• Standard MCP Lifecycle: Implements initialize, tools/list, and tools/call methods through dispatchRpc() to enable discovery and execution.
• Dynamic Control: Allows agents to modify routing strategies in real-time while maintaining read-only access to sensitive provider and usage data.

Frequently Asked Questions

What is the Model Context Protocol (MCP) in FreeLLMAPI?

The Model Context Protocol is a JSON-RPC 2.0 interface implemented in server/src/routes/mcp.ts that allows AI coding agents to introspect the FreeLLMAPI gateway's internal state. It exposes a registry of tools that agents can invoke to query model availability, provider health, and routing configurations without maintaining persistent connections.

How does authentication work for the MCP endpoint?

The MCP endpoint reuses the unified API key validation from the OpenAI-compatible /v1 endpoints. The authenticate() function accepts keys as Bearer tokens or x-api-key headers. Invalid authentication returns JSON-RPC error code -32001, consistent with the gateway's security model defined in server/src/routes/mcp.ts.

Which programming tools can interact with this MCP server?

Any MCP-compatible client can connect, including Claude Code, Cursor, and Cline. These agents follow the three-step protocol—initialize, list tools, and call tools—to discover and invoke introspection utilities like list_models or set_routing_strategy against the stateless /mcp endpoint.

Can agents modify gateway configuration through MCP introspection?

Yes, agents can modify runtime routing behavior through the set_routing_strategy tool, which invokes setRoutingStrategy() to change policies dynamically. However, the server restricts destructive changes; tools like provider_health and usage_summary provide read-only access to SQLite data without exposing raw credentials or allowing database modifications.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}