How oMLX Enforces Process Memory Limits to Prevent OOM Crashes

oMLX prevents out-of-memory (OOM) crashes by running a process-memory enforcer that continuously monitors the total Metal memory used by the MLX runtime and evicts or aborts models when usage exceeds a configurable soft limit.

The jundot/omlx repository implements a robust safeguard against GPU OOM errors through its ProcessMemoryEnforcer class. By tracking active memory allocations in real time and applying intelligent eviction policies, the system ensures that inference workloads remain within bounded process memory limits while maintaining service availability.

Monitoring Metal Memory via Asyncio Polling

The enforcement mechanism relies on a background task that periodically queries the MLX Metal allocator. In omlx/process_memory_enforcer.py, the enforcer calls mx.get_active_memory() every second (configurable via poll_interval) to obtain the current GPU memory footprint. This polling loop runs as an asyncio task, ensuring non-blocking operation while the inference engine serves requests. The retrieved value represents the total active memory held by the MLX runtime, which the enforcer compares against configured thresholds to trigger protective actions.

Configuring Soft and Hard Memory Limits

oMLX utilizes a two-tier limit system to balance protection against overhead. The soft limit (max_bytes) represents the user-configurable memory ceiling that triggers eviction behavior. The hard limit, calculated by _get_hard_limit_bytes in omlx/process_memory_enforcer.py (lines 109–119), is defined as the larger value between the soft limit and the system RAM minus 4 GiB. This calculation guarantees additional headroom for the pre-fill guard, ensuring that temporary allocation spikes during prompt processing do not breach the absolute physical boundary.

Propagating Limits to Inference Schedulers

When the enforcer initializes or when the limit changes dynamically, values propagate throughout the inference pipeline. The _propagate_memory_limit method (lines 151–165 in omlx/process_memory_enforcer.py) copies the current memory constraints into each model’s scheduler and its associated batch_generator. This propagation enables the scheduler to abort new requests that would exceed the limit before they allocate memory, preventing OOM conditions at the request ingress point rather than during allocation.

Eviction and Abort Strategies

When polled memory exceeds the soft limit, the enforcer acquires the EnginePool lock and executes a tiered eviction policy based on model availability and request state.

Evicting the Least-Recently-Used Model

If multiple models reside in memory, the enforcer identifies the least-recently-used (LRU) non-pinned model via EnginePool._find_lru_victim and unloads it repeatedly until usage drops below the threshold. During this process, the enforcer acquires the victim engine’s lock and invokes engine.abort_all_requests() (if implemented) to cancel any in-flight requests gracefully and release associated KV-cache allocations (lines 44–61).

Preserving Single-Model Availability

When only a single model remains loaded, the enforcer adopts a conservative approach to maintain service continuity. Rather than unloading the sole model—which would require expensive reallocation for subsequent requests—the system aborts all pending requests for that model but keeps the model loaded (lines 62–69). This strategy ensures that short-context requests can still be served without triggering new GPU allocations that might push the system over the limit.

Cancelling In-Flight Loading Operations

If no unloadable models exist but a loading operation is currently underway, the enforcer sets entry.abort_loading = True on the EnginePoolEntry to cancel the loading operation (lines 78–89). This prevents further memory growth from incomplete model loads when the system is already at capacity.

Graceful Shutdown

The stop() coroutine (lines 66–74 in omlx/process_memory_enforcer.py) ensures clean termination by cancelling the background polling task and clearing internal flags. This prevents stray timers from continuing to monitor memory after the service shuts down, eliminating race conditions during the shutdown sequence.

Implementation Example

The following example demonstrates instantiating and configuring the enforcer:

import asyncio
from omlx.process_memory_enforcer import ProcessMemoryEnforcer
from omlx.engine_pool import EnginePool

# Create the engine pool (example – actual creation may differ)

engine_pool = EnginePool()

# Define a soft memory limit of 8 GiB

MAX_MEMORY_BYTES = 8 * 1024**3

# Instantiate the enforcer with pre-fill guard enabled

enforcer = ProcessMemoryEnforcer(
    engine_pool=engine_pool,
    max_bytes=MAX_MEMORY_BYTES,
    poll_interval=1.0,          # check every second

    prefill_memory_guard=True
)

# Start the background enforcement loop

enforcer.start()

# Dynamically adjust the limit (e.g., after loading a new model)

enforcer.max_bytes = 10 * 1024**3

# Graceful shutdown when terminating the service

await enforcer.stop()

Key Source Files

• omlx/process_memory_enforcer.py – Implements the async polling loop, limit propagation, LRU eviction, and request abort logic that enforces the process-level memory ceiling.
• omlx/memory_monitor.py – Provides GPU-memory-usage statistics and KV-cache estimations; used by the enforcer to report status and by the scheduler for pre-fill checks.
• omlx/engine_pool.py – Manages the collection of loaded models (EnginePool._entries) and supplies the LRU-victim selection logic.
• omlx/settings.py – Supplies get_system_memory() and global settings referenced by the enforcer for hard-limit calculations.

Summary

• Continuous monitoring – A background asyncio task polls mx.get_active_memory() every second to track Metal GPU usage in real time.
• Tiered limits – The soft limit triggers evictions, while the hard limit (system RAM minus 4 GiB) provides absolute protection against physical OOM.
• Proactive propagation – Memory constraints propagate to model schedulers to prevent new requests from exceeding limits before allocation.
• Intelligent eviction – The system unloads LRU non-pinned models first, aborts requests for single-model scenarios while keeping the model resident, and cancels loading operations when necessary.
• Clean lifecycle – The stop() coroutine ensures the monitoring task terminates gracefully without resource leaks.

Frequently Asked Questions

What is the difference between the soft limit and hard limit in oMLX?

The soft limit (max_bytes) is the user-configurable threshold that triggers eviction behaviors when exceeded. The hard limit, calculated as the maximum of the soft limit or system RAM minus 4 GiB, represents a physical safety ceiling that ensures the pre-fill guard has sufficient headroom to prevent absolute OOM conditions.

How does oMLX decide which model to evict when memory is exhausted?

The enforcer selects the least-recently-used (LRU) non-pinned model from the EnginePool. It repeatedly unloads LRU victims until memory drops below the soft limit, ensuring that frequently accessed models remain in cache while idle models are reclaimed.

What happens to active requests when the memory limit is exceeded?

For multi-model scenarios, active requests on evicted models are cancelled via engine.abort_all_requests(), releasing their KV-cache allocations. When only one model remains, the enforcer aborts all pending requests for that model but preserves the model in memory to avoid costly reloading for short-context queries.

How frequently does the enforcer check memory usage?

The default polling interval is 1 second, controlled by the poll_interval parameter passed to ProcessMemoryEnforcer. This interval represents a trade-off between responsiveness to memory spikes and the overhead of Metal API calls.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}