How to Use the oMLX Anthropic Messages API Endpoint: A Complete Implementation Guide

oMLX exposes a fully Anthropic-compatible Messages API at /v1/messages that leverages the AnthropicAdapter class to translate requests into internal formats and returns either standard JSON responses or Server-Sent Events (SSE) streams matching the official Anthropic specification.

The open-source jundot/omlx repository provides a complete implementation of the Anthropic Messages API, enabling you to serve local language models behind a familiar interface. By utilizing the adapter pattern in omlx/api/adapters/anthropic.py, the system converts incoming Anthropic-style payloads into unified InternalRequest objects, processes them through the inference engine, and formats outputs to maintain full compatibility with existing Anthropic SDKs and tools.

Architecture of the Anthropic Adapter

The core of this functionality resides in the AnthropicAdapter class, which handles bidirectional translation between the Anthropic API schema and oMLX’s internal representation.

Request Parsing and Internal Conversion

When you POST to the Messages endpoint, AnthropicAdapter.parse_request receives a Pydantic-validated MessagesRequest object and initiates the conversion pipeline:

1. Message Translation – The adapter calls convert_anthropic_to_internal to transform the list of messages (including optional system messages) into internal dictionaries.
2. Tool Handling – If the request includes a tools parameter, convert_anthropic_tools_to_internal processes these definitions for the inference engine.
3. Request Assembly – The method constructs an InternalRequest containing normalized parameters (max_tokens, temperature, top_p/k, streaming flags, stop sequences, model name, and request ID).

According to the source code in omlx/api/adapters/anthropic.py (lines 51-90), this abstraction allows the inference engine to remain agnostic while supporting Anthropic-specific features like tool use and structured system prompts.

Response Generation

For non-streaming responses, AnthropicAdapter.format_response receives an InternalResponse from the engine and invokes convert_internal_to_anthropic_response to generate JSON output matching the Anthropic schema (lines 92-107).

For streaming responses, AnthropicAdapter.format_stream_chunk constructs the specific SSE event sequence required by the Anthropic protocol:

• message_start and content_block_start (initial metadata)
• text_delta or input_json_delta (content chunks)
• content_block_stop, message_delta, and message_stop (termination signals)

Unlike OpenAI’s implementation, Anthropic streaming does not use a [DONE] marker. Instead, AnthropicAdapter.format_stream_end returns an empty string, relying on the message_stop event as the stream terminator (lines 163-174).

How to Call the oMLX Anthropic Messages API Endpoint

The server typically exposes the endpoint at http://localhost:8000/v1/messages when running locally.

Standard POST Request (Non-Streaming)

Send a JSON payload to receive a complete response object:

import requests

url = "http://localhost:8000/v1/messages"

payload = {
    "model": "claude-2.0",
    "messages": [
        {"role": "user", "content": "Write a haiku about sunrise."}
    ],
    "max_tokens": 256,
    "temperature": 0.7,
    "stream": False,
    "stop_sequences": []
}

resp = requests.post(url, json=payload)
print(resp.json())

Processing flow:

• The FastAPI route in omlx/server.py forwards the request to AnthropicAdapter.parse_request.
• After inference, AnthropicAdapter.format_response returns a JSON object containing id, type, completion, stop_reason, and usage statistics.

Streaming with Server-Sent Events

Enable streaming by setting stream: True and processing SSE events:

import requests
import sseclient

url = "http://localhost:8000/v1/messages"
payload = {
    "model": "claude-2.0",
    "messages": [{"role": "user", "content": "Explain recursion in plain English."}],
    "max_tokens": 512,
    "temperature": 0.6,
    "stream": True
}

r = requests.post(url, json=payload, stream=True)
client = sseclient.SSEClient(r)

for event in client.events():
    print(event.data)

Event sequence: The stream emits standardized Anthropic events produced by AnthropicAdapter.format_stream_chunk (lines 116-161 in omlx/api/adapters/anthropic.py). Your client receives progressive text_delta updates until the final message_stop event signals completion.

Direct Python Adapter Integration

For unit testing or custom routing logic, instantiate the adapter directly:

from omlx.api.adapters.anthropic import AnthropicAdapter
from omlx.api.anthropic_models import MessagesRequest

adapter = AnthropicAdapter()

request = MessagesRequest(
    model="claude-2.0",
    messages=[{"role": "user", "content": "What is the capital of France?"}],
    max_tokens=128,
    temperature=0.5,
    stream=False,
)

internal_req = adapter.parse_request(request)

# Pass internal_req to inference engine...

# Then format the result:

# response = adapter.format_response(internal_response, request)

This approach bypasses HTTP overhead and uses the Pydantic models defined in omlx/api/anthropic_models.py for type-safe request construction.

Error Handling and Tool Support

The adapter implements Anthropic-compatible error formatting through AnthropicAdapter.create_error_response for standard JSON errors and format_error_event for SSE error streams (lines 177-194). Errors follow the standard Anthropic error structure with type and message fields.

For tool calling, include the tools parameter in your request. The adapter transforms these using convert_anthropic_tools_to_internal (lines 74-78), enabling function calling workflows compatible with the Anthropic API specification.

Summary

• Endpoint Location: POST requests to /v1/messages handled by the FastAPI server in omlx/server.py.
• Core Adapter: AnthropicAdapter in omlx/api/adapters/anthropic.py manages bidirectional payload conversion.
• Streaming Protocol: Uses Anthropic-specific SSE events (message_start, text_delta, message_stop) without a [DONE] marker, formatted by format_stream_chunk.
• Direct Access: Import AnthropicAdapter and MessagesRequest for programmatic integration without HTTP.
• Tool Support: Automatic conversion of Anthropic tool schemas via convert_anthropic_tools_to_internal.

Frequently Asked Questions

What is the exact URL path for the oMLX Anthropic Messages API?

The endpoint is mounted at /v1/messages by default when running the oMLX server locally on port 8000, making the full URL http://localhost:8000/v1/messages. The FastAPI router delegates these requests to the AnthropicAdapter class based on the provider configuration in your server setup.

Does the oMLX Anthropic endpoint support function calling and tools?

Yes. When you include a tools array in your request payload, AnthropicAdapter.parse_request automatically invokes convert_anthropic_tools_to_internal to transform the tool definitions. The adapter handles both the input tool schemas and the streaming input_json_delta events when the model generates tool call arguments.

How does oMLX handle streaming differently than OpenAI-compatible endpoints?

oMLX implements Anthropic’s native SSE protocol using specific event types (message_start, content_block_start, text_delta, etc.) produced by format_stream_chunk. Unlike OpenAI’s streaming format which terminates with a [DONE] marker, the Anthropic adapter uses format_stream_end to return an empty string, signaling completion solely through the message_stop event type.

Can I use the official Anthropic Python SDK with oMLX?

Yes. Because oMLX’s AnthropicAdapter returns responses that match the official Anthropic JSON schema and SSE format, you can point the Anthropic SDK to your local oMLX server by setting the base URL to http://localhost:8000. The SDK will function normally, parsing the standardized responses generated by convert_internal_to_anthropic_response without requiring code changes.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}