# How the MCP Server Enables Agent Introspection in FreeLLMAPI

> Discover how the MCP server in FreeLLMAPI facilitates agent introspection by enabling queries to runtime router state. Access model availability, provider health, and routing strategies via a JSON-RPC interface.

- Repository: [Tashfeen/freellmapi](https://github.com/tashfeenahmed/freellmapi)
- Tags: internals
- Published: 2026-08-31

---

**The FreeLLMAPI gateway implements a stateless MCP (Model Context Protocol) server at `/mcp` that allows coding agents to query runtime router state—such as available models, provider health, and routing strategies—through a JSON-RPC 2.0 interface without maintaining persistent sessions.**

The Model Context Protocol is an open standard designed to let AI assistants interact with external systems. According to the FreeLLMAPI source code, the gateway exposes a lightweight JSON-RPC façade over its internal router state in [`server/src/routes/mcp.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/routes/mcp.ts), enabling agents like Claude Code, Cursor, and Cline to introspect and even modify gateway behavior dynamically.

## Stateless JSON-RPC Transport

Agents POST single JSON-RPC 2.0 messages to the **`/mcp`** endpoint. The implementation explicitly rejects batching per protocol version **2025-06-18** and returns plain JSON objects. The server is strictly stateless—GET or DELETE requests receive HTTP 405 errors explaining that no session storage is maintained.

Authentication mirrors the OpenAI-compatible `/v1` endpoints. The `authenticate()` function validates Bearer tokens or `x-api-key` headers against the unified API key system. Invalid keys return JSON-RPC error **-32001**.

## The Seven Introspection Tools

The `TOOLS` constant in [`server/src/routes/mcp.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/routes/mcp.ts) registers seven utilities, each defining a `name`, description, JSON-Schema `inputSchema`, and handler function.

### Model Discovery with `list_models`

The `list_models` tool invokes `buildModelListing()` from [`server/src/services/model-listing.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/model-listing.ts) to compile a catalog of free models. The response includes context windows, tool support flags, and per-platform parameters, formatted as structured JSON.

### Provider Health Monitoring with `provider_health`

This tool queries the SQLite database via `getDb()` to compute provider status counts, active rate-limit cool-downs, and usable model tallies. Agents use this to detect provider outages before routing requests.

### Usage Analytics with `usage_summary`

Aggregating hourly statistics from the database, `usage_summary` accepts time ranges (`24h`, `7d`, `30d`) and returns request totals, token counts, success rates, and top-traffic models via the `usageSummary()` handler.

### Routing Control with `routing_info` and `set_routing_strategy`

The `routing_info` tool exposes the active strategy and top-scored fallback models. Through `set_routing_strategy`, agents invoke `setRoutingStrategy()` to switch policies dynamically without restarting the gateway.

### Performance Metrics with `cache_stats` and `compression_stats`

`cache_stats` retrieves hit/miss counters from [`server/src/services/cache.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/cache.ts), while `compression_stats` pulls metrics from [`server/src/services/compression/stats.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/services/compression/stats.ts). These reveal quota savings from the compression engine and caching layer.

## The Introspection Flow: Three-Step Protocol

The `dispatchRpc()` function orchestrates interactions through a standardized lifecycle:

1. **Initialize**: Agents call `initialize` to receive protocol capabilities and version `2025-06-18`.
2. **List Tools**: Agents request `tools/list` to discover available utilities and their input schemas.
3. **Call Tools**: Agents invoke `tools/call` with a tool name and arguments, triggering the specific handler and returning a JSON-RPC result envelope containing tool-specific data.

```json
POST /mcp
{
  "jsonrpc": "2.0",
  "id": 1,
  "method": "initialize",
  "params": {}
}

```

```json
POST /mcp
{
  "jsonrpc": "2.0",
  "id": 2,
  "method": "tools/list",
  "params": {}
}

```

```json
POST /mcp
{
  "jsonrpc": "2.0",
  "id": 3,
  "method": "tools/call",
  "params": {
    "name": "set_routing_strategy",
    "arguments": { "strategy": "smartest" }
  }
}

```

## Stateless Architecture and Implementation

Registered in [`server/src/app.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/app.ts), the MCP router maintains zero session state. Each request is processed independently by `dispatchRpc()`, which routes to the appropriate handler based on the method field. This design eliminates connection overhead while providing full runtime visibility into routing decisions, provider status, and optimization metrics stored in the SQLite backend.

## Summary

- **Stateless JSON-RPC Endpoint**: The `/mcp` route in [`server/src/routes/mcp.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/routes/mcp.ts) processes single-message JSON-RPC 2.0 requests, rejecting batching and returning 405 errors for non-POST methods.
- **Unified Authentication**: Uses the existing API key system from OpenAI-compatible endpoints, returning error code -32001 for invalid credentials via the `authenticate()` function.
- **Seven Introspection Tools**: Exposes `list_models`, `provider_health`, `usage_summary`, `routing_info`, `set_routing_strategy`, `cache_stats`, and `compression_stats` through the `TOOLS` registry.
- **Standard MCP Lifecycle**: Implements `initialize`, `tools/list`, and `tools/call` methods through `dispatchRpc()` to enable discovery and execution.
- **Dynamic Control**: Allows agents to modify routing strategies in real-time while maintaining read-only access to sensitive provider and usage data.

## Frequently Asked Questions

### What is the Model Context Protocol (MCP) in FreeLLMAPI?

The Model Context Protocol is a JSON-RPC 2.0 interface implemented in [`server/src/routes/mcp.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/routes/mcp.ts) that allows AI coding agents to introspect the FreeLLMAPI gateway's internal state. It exposes a registry of tools that agents can invoke to query model availability, provider health, and routing configurations without maintaining persistent connections.

### How does authentication work for the MCP endpoint?

The MCP endpoint reuses the unified API key validation from the OpenAI-compatible `/v1` endpoints. The `authenticate()` function accepts keys as Bearer tokens or `x-api-key` headers. Invalid authentication returns JSON-RPC error code -32001, consistent with the gateway's security model defined in [`server/src/routes/mcp.ts`](https://github.com/tashfeenahmed/freellmapi/blob/main/server/src/routes/mcp.ts).

### Which programming tools can interact with this MCP server?

Any MCP-compatible client can connect, including Claude Code, Cursor, and Cline. These agents follow the three-step protocol—initialize, list tools, and call tools—to discover and invoke introspection utilities like `list_models` or `set_routing_strategy` against the stateless `/mcp` endpoint.

### Can agents modify gateway configuration through MCP introspection?

Yes, agents can modify runtime routing behavior through the `set_routing_strategy` tool, which invokes `setRoutingStrategy()` to change policies dynamically. However, the server restricts destructive changes; tools like `provider_health` and `usage_summary` provide read-only access to SQLite data without exposing raw credentials or allowing database modifications.