# How to Troubleshoot Common MLX-VLM Issues: A Complete Guide

> Troubleshoot common MLX-VLM errors with this complete guide. Resolve runtime issues by validating inputs, flags, and model configurations for smooth operation.

- Repository: [Prince Canuma/mlx-vlm](https://github.com/Blaizzy/mlx-vlm)
- Tags: how-to-guide
- Published: 2026-04-05

---

**Most MLX-VLM runtime errors stem from mismatched inputs, missing flags for quantization features, or incompatible model configurations, and can be resolved by validating file paths, checking model compatibility, and using the correct cache and quantization parameters.**

Blaizzy/mlx-vlm is an open-source vision-language framework that unifies vision encoders, large language models, and optional audio/video pipelines with performance optimizations like KV-cache quantization and LoRA adapters. When you troubleshoot common MLX-VLM issues, you'll typically encounter problems in model loading, input preprocessing, or memory management. This guide maps specific error messages to their exact source code locations and provides actionable fixes.

## Model Loading and Configuration Errors

### HTTP 500 Errors and Remote Code Execution

Model loading failures originate in `mlx_vlm.utils.load` (lines 63‑70), which orchestrates loading the model config, weights, and processor. If the server wrapper cannot initialize the model, it raises `HTTPException 500: Failed to load model` at line 157 of [`mlx_vlm/server.py`](https://github.com/Blaizzy/mlx-vlm/blob/main/mlx_vlm/server.py).

To resolve these errors:

- **Verify the model identifier** exists on Hugging Face and you have network access.
- **Set environment variables** if the repository contains custom code: `export MLX_TRUST_REMOTE_CODE=true` (see server options at lines 4‑10).
- **Check quantization compatibility**: If using `mxfp8` or `nvfp4` models, pass `--quantize-activations` or `quantize_activations=True` to `load()` as documented in the docstring (lines 84‑90).

```python
from mlx_vlm import load

# Safe loading with activation quantization and LoRA

model, processor = load(
    "mlx-community/Qwen2-VL-2B-Instruct-4bit",
    adapter_path="/path/to/lora/adapter",  # Optional LoRA

    quantize_activations=True,             # Required for nvfp4/mxfp8 on CUDA

    trust_remote_code=True                 # For custom processor code

)

```

### Activation Quantization Validation

The `load()` function strictly validates that `quantize_activations` only works for models exported with `nvfp4` or `mxfp8` quantization. If you attempt to use this flag on an incompatible model, the library raises `ValueError: Activation quantisation is not supported for this model`.

**Fix**: Confirm your model was exported with a supported CUDA quantization mode before enabling `-qa` or `--quantize-activations` in the CLI.

## Input Processing Failures

### Image Loading and Validation

Image loading errors surface in `mlx_vlm.utils.load_image` (lines 46‑55), which validates source types and raises `ValueError: Failed to load image from …` for unsupported inputs.

Common causes and solutions:

- **Unreachable URLs or invalid paths**: Ensure the file exists locally or the URL returns a valid image.
- **Malformed data-URIs**: Verify the URI contains the comma separator: `data:image/png;base64,…`.
- **BytesIO objects**: Confirm the stream contains valid image data before passing it to the loader.

```python
from mlx_vlm.utils import load_image

try:
    image = load_image("https://example.com/picture.jpg")
except ValueError as exc:
    raise RuntimeError(f"Could not load the image: {exc}") from exc

```

### Unsupported Audio and Video Formats

The framework validates media formats in `mlx_vlm.video_generate.load_video` (line 121) and `mlx_vlm.utils.load_audio` (line 966), raising `ValueError` for unknown extensions.

**Supported formats**:
- **Images**: `.jpg`, `.png`
- **Audio**: `.wav`, `.mp3`
- **Video**: Common containers like `.mp4`

Convert media to supported formats before processing, and use the `--audio` or `--video` CLI flags with correct paths.

## Quantization and Memory Errors

### KV-Cache Quantization with TurboQuant

TurboQuant errors originate in `mlx_vlm.turboquant._validate_bits` (lines 46‑55), which enforces that `kv_bits` ≥ 1 and accepts only integer or half-integer widths. The helper `turboquant_enabled` (lines 58‑65) determines if the backend should activate.

If you encounter `ValueError: TurboQuant requires kv_bits >= 1` or `TurboQuant currently supports integer and .5 bit‑widths`:

1. Use supported bit-widths: **2**, **3**, **3.5**, or **4**.
2. Pass both `--kv-bits` and `--kv-quant-scheme turboquant`.
3. Note that models using `RotatingKVCache` automatically disable TurboQuant.

```python
from mlx_vlm import load, generate

model, processor = load(
    "mlx-community/Qwen3.5-4B-4bit",
    kv_bits=3.5,                 # Half-integer width supported

    kv_quant_scheme="turboquant"
)

output = generate(
    model, processor, "Summarize this document",
    max_tokens=512,
    kv_bits=3.5,
    kv_quant_scheme="turboquant"
)

```

### Vision Feature Cache Misses and OOM

The `mlx_vlm.vision_cache.VisionFeatureCache` stores projected image features keyed by path, URL, or raw image data hash. Cache misses cause re-encoding overhead, while unbounded growth leads to memory spikes.

**Optimization strategies**:
- **Reuse cache instances** across `stream_generate` calls rather than creating new ones.
- **Increase cache size** if processing many images: `VisionFeatureCache(max_size=64)`.
- **Clear caches** when swapping models: `cache.clear()`.

```python
from mlx_vlm import load, stream_generate, VisionFeatureCache, apply_chat_template

model, processor = load("mlx-community/Qwen2-VL-2B-Instruct-4bit")
cache = VisionFeatureCache(max_size=64)  # Increase from default 20 entries

image_path = "cat.png"
prompt1 = apply_chat_template(processor, model.config, "Describe the image.", num_images=1)

# First turn: cache miss, image encoded

for chunk in stream_generate(model, processor, prompt1, image=[image_path], vision_cache=cache):
    print(chunk.text, end="")

# Second turn: cache hit, no vision re-encoding

prompt2 = apply_chat_template(processor, model.config, "What color is the cat?", num_images=1)
for chunk in stream_generate(model, processor, prompt2, image=[image_path], vision_cache=cache):
    print(chunk.text, end="")

```

## Adapter and Processor Issues

### Missing LoRA Adapters

The `mlx_vlm.utils.apply_lora_layers` function raises `FileNotFoundError` when the adapter path does not exist (see [`trainer/utils.py`](https://github.com/Blaizzy/mlx-vlm/blob/main/trainer/utils.py) line 191). The error message indicates the specific missing path.

**Resolution**:
- Verify the adapter directory contains `adapter.safetensors`.
- Pass the correct `--adapter-path` to the CLI or `adapter_path` parameter to `load()`.

### Processor Configuration Errors

Many functions depend on the processor having an `image_processor` attribute (see `load()` lines 15‑17). If a model lacks a processor, `load_processor` raises `ValueError: Processor missing feature_extractor for audio prep.`

**Fix**: Ensure you load models that provide official processors, or implement `ProcessorMixin` and enable `--trust-remote-code` for custom implementations.

## Video Generation Debugging

Video processing fails in `mlx_vlm.video_generate.load_video` (lines 209‑228) when frame extraction fails or codecs are unsupported. Symptoms include `ValueError: Cannot open video: …` or `No frames read from the video.`

**Troubleshooting steps**:
- **Install ffmpeg**: Required for non-MP4 codecs and many video containers.
- **Validate hardware limits**: Adjust `--max-pixels` and `--fps` arguments to match your system's capabilities.

```python
from mlx_vlm.video_generate import load_video, generate_video

try:
    video_frames = load_video("sample.mp4")
except ValueError as exc:
    raise RuntimeError(f"Video loading failed: {exc}. Ensure ffmpeg is installed.") from exc

output = generate_video(
    model, processor,
    prompt="Describe the video.",
    video_frames=video_frames,
    max_tokens=200
)

```

## Summary

- **Model loading errors** in [`mlx_vlm/utils.py`](https://github.com/Blaizzy/mlx-vlm/blob/main/mlx_vlm/utils.py) typically require checking Hugging Face access, enabling `trust_remote_code`, or matching quantization flags to model types.
- **Input validation failures** trace to `load_image` and media loaders; ensure URLs are reachable and file formats match supported extensions.
- **KV-cache quantization** supports only specific bit-widths (2, 3, 3.5, 4) as enforced in [`turboquant.py`](https://github.com/Blaizzy/mlx-vlm/blob/main/turboquant.py).
- **Vision cache management** prevents OOM by reusing `VisionFeatureCache` instances and adjusting `max_size`.
- **LoRA and processor errors** resolve by verifying file paths at [`trainer/utils.py`](https://github.com/Blaizzy/mlx-vlm/blob/main/trainer/utils.py) line 191 and ensuring processor attributes exist.

## Frequently Asked Questions

### Why do I get "HTTPException 500: Failed to load model" when starting the server?

This error originates in [`mlx_vlm/server.py`](https://github.com/Blaizzy/mlx-vlm/blob/main/mlx_vlm/server.py) line 157 when `mlx_vlm.utils.load` cannot initialize the model. Verify the model identifier exists on Hugging Face, ensure you have network connectivity, and set `MLX_TRUST_REMOTE_CODE=true` if the repository contains custom Python code.

### What bit-widths does TurboQuant support for KV-cache quantization?

According to the `_validate_bits` implementation in [`mlx_vlm/turboquant.py`](https://github.com/Blaizzy/mlx-vlm/blob/main/mlx_vlm/turboquant.py) lines 46‑55, TurboQuant supports **2**, **3**, **3.5**, and **4** bits. Attempting to use other values raises `ValueError` indicating that only integer and half-integer widths are accepted.

### How do I prevent out-of-memory errors when processing batches of images?

Use the `VisionFeatureCache` class from [`mlx_vlm/vision_cache.py`](https://github.com/Blaizzy/mlx-vlm/blob/main/mlx_vlm/vision_cache.py) to store projected features between calls. Reuse the same cache instance across `stream_generate` invocations, increase the default `max_size` from 20 to a higher value (e.g., 64), and call `cache.clear()` when switching models or ending long sessions.

### Why does my video file fail to load with "Cannot open video"?

The `load_video` function in [`mlx_vlm/video_generate.py`](https://github.com/Blaizzy/mlx-vlm/blob/main/mlx_vlm/video_generate.py) lines 209‑228 raises this error when it cannot extract frames, usually due to missing `ffmpeg` installation or unsupported codecs. Install `ffmpeg` on your system and convert the video to a common container like `.mp4` before processing.