omlx

Explore the oMLX cache stack architecture. Discover its two-tier caching with in-memory LRU and SSD persistence to speed up vision feature tensor computation.

Learn how to set up model aliases in oMLX to use custom API-visible names. Decouple storage layout from client identifiers for better API management.

Integrate oMLX with OpenCode OpenClaw or Codex effortlessly. This guide shows how to run the oMLX inference server and use Python classes to auto configure your tools for seamless integration.

Learn how LRU eviction in oMLX efficiently manages GPU memory for multi-model serving, evicting least-recently-used models to make space for new requests.

Learn how to configure oMLX hot cache size and cold cache size using environment variables or CLI flags for optimal performance. Set RAM and disk allocations effectively.

Learn how to run performance benchmarks in oMLX using its built-in utility. Measure TTFT, generation throughput, and continuous batching performance with ease.

Easily download HuggingFace models directly within the oMLX admin dashboard. Our FastAPI integration simplifies model management for your projects.

Implement the Anthropic Messages API endpoint with oMLX. Discover how to use this compatible API for seamless integration and JSON or SSE stream responses. Get the complete guide.

Use oMLX to easily replace the OpenAI API with local MLX models. Redirect your OpenAI client to local MLX models by changing the base URL.

Learn to customize chat template kwargs per model in oMLX. Control Jinja parameters, merge settings, and enforce policies with forced_ct_kwargs for robust chat management in your Jundot/omlx projects.

Force oMLX model type detection (LLM vs VLM) with model_type_override. Update the engine pool instantly without server restarts and gain full control over your model configurations.

Learn how oMLX enforces process memory limits to prevent OOM crashes. Discover its continuous monitoring and eviction strategies for stable MLX runtime.