# How to Set Up MCP (Model Context Protocol) with an oMLX Server

> Learn how to set up MCP with an oMLX server by creating a config file and launching the server. Expose model inference easily.

- Repository: [Jun Kim/omlx](https://github.com/jundot/omlx)
- Tags: how-to-guide
- Published: 2026-05-11

---

**Setting up MCP with oMLX requires creating a JSON or YAML configuration file, pointing the server to it via the** `OMLX_MCP_CONFIG` **environment variable or** `--mcp-config` **CLI flag, and launching the server with** `omlx serve` **to expose model inference capabilities over STDIO or SSE transports.**

The **Model Context Protocol (MCP)** is the standardized mechanism that the [jundot/omlx](https://github.com/jundot/omlx) repository uses to bridge external clients with model inference, tool calling, and streaming completions. By implementing the transport layer defined in `omlx/mcp/`, the server can route OpenAI-compatible API requests through configurable backends. This guide walks through the exact configuration steps based on the current source implementation.

## Create the MCP Configuration File

The server expects a configuration file that describes the model, transport protocol, and optional tool definitions. This file drives the initialization logic in [`omlx/mcp/config.py`](https://github.com/jundot/omlx/blob/main/omlx/mcp/config.py), where the `MCPConfig` dataclass validates all inputs.

A minimal configuration ([`mcp.json`](https://github.com/jundot/omlx/blob/main/mcp.json)) requires the following fields:

```json
{
  "model_name": "llama-3.1-8b-instruct",
  "max_context_len": 8192,
  "transport": "stdio",
  "server_name": "default",
  "tools": [
    {
      "server_name": "default",
      "name": "search",
      "description": "Perform a web search and return the top result."
    }
  ],
  "default_temperature": 0.7,
  "default_top_p": 0.95
}

```

Key fields defined in [`omlx/mcp/types.py`](https://github.com/jundot/omlx/blob/main/omlx/mcp/types.py) include:

- **`model_name`** – Must match an entry registered in [`omlx/model_registry.py`](https://github.com/jundot/omlx/blob/main/omlx/model_registry.py).
- **`transport`** – Accepts `"stdio"` for subprocess communication via newline-delimited JSON, or `"sse"` for Server-Sent Events over HTTP.
- **`tools`** – Optional array of `MCPTool` definitions that expose callable functions to the client.

Save this file to `~/.config/omlx/mcp.json` (or use the `.yaml` extension if preferred). The `MCPConfig.from_path()` method in [`omlx/mcp/config.py`](https://github.com/jundot/omlx/blob/main/omlx/mcp/config.py) handles parsing and validation for both formats.

## Point the Server to Your Configuration

The server resolves the configuration location using a strict precedence order implemented in the loader:

1. Path supplied via `--mcp-config <path>` on the CLI.
2. Path specified in the `OMLX_MCP_CONFIG` environment variable.
3. Default locations: `~/.config/omlx/mcp.json` or `~/.config/omlx/mcp.yaml`.

Set the environment variable before launching:

```bash
export OMLX_MCP_CONFIG=$HOME/.config/omlx/mcp.json

```

Alternatively, pass the path directly when starting the server:

```bash
omlx serve --mcp-config /custom/path/mcp.yaml

```

If the file is missing or malformed, `MCPConfig.from_path()` raises a validation error referencing the specific schema violation found in [`omlx/mcp/types.py`](https://github.com/jundot/omlx/blob/main/omlx/mcp/types.py).

## Launch the oMLX Server

With the configuration in place, start the server:

```bash
omlx serve

```

During initialization, the server executes the following sequence defined in [`omlx/mcp/manager.py`](https://github.com/jundot/omlx/blob/main/omlx/mcp/manager.py) and [`omlx/mcp/executor.py`](https://github.com/jundot/omlx/blob/main/omlx/mcp/executor.py):

- Invokes `MCPConfig.from_dict()` to instantiate the configuration object.
- Creates an `MCPServerStatus` instance with `state=CONNECTING` and the selected `MCPTransport` enum.
- Spawns the appropriate executor: `MCPExecutorSTDIO` for subprocess management or `MCPExecutorSSE` for HTTP event streaming.

For **STDIO** transport, the server launches a subprocess (e.g., `python -m omlx.mlx`) and communicates via JSON lines over stdin/stdout. For **SSE** transport, the server exposes an endpoint at `/v1/mcp/events` that clients subscribe to for streaming partial completions.

## Connect a Client and Issue Requests

Once running, use the `MCPClient` class from [`omlx/mcp/client.py`](https://github.com/jundot/omlx/blob/main/omlx/mcp/client.py) to interact with the server. This façade wraps the generic request building logic in [`omlx/api/openai_models.py`](https://github.com/jundot/omlx/blob/main/omlx/api/openai_models.py) and translates standard OpenAI-style calls into the MCP transport layer.

```python
from omlx.mcp.client import MCPClient

# Automatically reads OMLX_MCP_CONFIG if set

client = MCPClient()

response = client.chat_completion(
    model="llama-3.1-8b-instruct",
    messages=[{"role": "user", "content": "Write a haiku about AI."}],
    temperature=0.7,
)

print(response["choices"][0]["message"]["content"])

```

For **tool-calling**, include the tool definition in your request. The client maps `MCPTool` definitions to the OpenAI-compatible format expected by the executor:

```python
response = client.chat_completion(
    model="llama-3.1-8b-instruct",
    messages=[
        {"role": "user", "content": "What is 7 * 6? Use the calc tool."}
    ],
    tool_calls=[{"name": "calc", "arguments": {"expr": "7*6"}}],
)

```

To consume **SSE streams** directly without the client wrapper:

```python
import requests

url = "http://localhost:8000/v1/mcp/events?model=llama-3.1-8b-instruct"
with requests.get(url, stream=True) as r:
    for line in r.iter_lines():
        if line:
            event = line.decode('utf-8')
            print(event)

```

## Summary

- **MCP** standardizes how oMLX exposes inference capabilities to external clients through a configuration-driven architecture.
- Create a JSON or YAML file defining `model_name`, `transport` (`stdio` or `sse`), and optional `tools`.
- Point the server to this file using `OMLX_MCP_CONFIG`, `--mcp-config`, or place it at `~/.config/omlx/mcp.json`.
- Launch with `omlx serve` to instantiate `MCPExecutorSTDIO` or `MCPExecutorSSE` as defined in [`omlx/mcp/executor.py`](https://github.com/jundot/omlx/blob/main/omlx/mcp/executor.py).
- Use `MCPClient` from [`omlx/mcp/client.py`](https://github.com/jundot/omlx/blob/main/omlx/mcp/client.py) for high-level API calls, or connect directly to SSE endpoints for streaming.

## Frequently Asked Questions

### What is the difference between STDIO and SSE transport in oMLX MCP?

**STDIO** spawns the model as a subprocess and communicates via newline-delimited JSON over stdin/stdout, making it ideal for local, process-isolated deployments. **SSE** (Server-Sent Events) exposes an HTTP endpoint that streams partial completions to connected clients, better suited for network-distributed or browser-based consumers. The executor selection is controlled by the `transport` field in your MCP configuration file, processed by `MCPExecutorSTDIO` and `MCPExecutorSSE` classes in [`omlx/mcp/executor.py`](https://github.com/jundot/omlx/blob/main/omlx/mcp/executor.py).

### Can I use YAML instead of JSON for the MCP configuration?

Yes. The `MCPConfig.from_path()` method in [`omlx/mcp/config.py`](https://github.com/jundot/omlx/blob/main/omlx/mcp/config.py) automatically detects the file extension and parses accordingly. Save your configuration as `~/.config/omlx/mcp.yaml` and the loader will validate it against the same `MCPConfig` schema defined in [`omlx/mcp/types.py`](https://github.com/jundot/omlx/blob/main/omlx/mcp/types.py) without requiring any syntax changes to the key names or structure.

### Why does the server fail to start with an "Invalid model_name" error?

The `model_name` value in your MCP configuration must exactly match a model registered in the internal model registry (typically [`omlx/model_registry.py`](https://github.com/jundot/omlx/blob/main/omlx/model_registry.py)). If you specify a custom or local model not yet registered, the validation logic in `MCPConfig.from_dict()` will reject the configuration. Verify available models by checking the registry or consult the repository documentation for the correct identifier strings.

### How do I enable tool-calling for my oMLX server?

Define the `tools` array in your MCP configuration file with objects containing `server_name`, `name`, and `description` fields that map to the `MCPTool` dataclass in [`omlx/mcp/types.py`](https://github.com/jundot/omlx/blob/main/omlx/mcp/types.py). When using `MCPClient`, pass the `tool_calls` parameter in your request; the client will format these according to the OpenAI tool-calling specification and route them through the MCP layer to the executor backend.