# Which Models Are Supported by MLX-VLM? Complete List and Usage Guide

> Discover which models MLX-VLM supports including Qwen2-VL, Qwen2.5-VL, Idefics3, and LLaVA. Get the complete list and usage guide for vision-language tasks on Apple Silicon.

- Repository: [Prince Canuma/mlx-vlm](https://github.com/Blaizzy/mlx-vlm)
- Tags: api-reference
- Published: 2026-04-05

---

**MLX-VLM officially supports four core vision-language architectures—Qwen2-VL, Qwen2.5-VL, Idefics3, and LLaVA—for image, video, and multi-modal chat inference on Apple Silicon.**

The Blaizzy/mlx-vlm repository delivers optimized inference for vision-language models (VLMs) using Apple's MLX framework. Understanding exactly which models are supported by MLX-VLM is essential for developers building multimodal applications on macOS and iOS devices. This guide breaks down the officially supported architectures, provides runnable code examples for each, and explains how the library validates model compatibility through its source code.

## Officially Supported Models

According to the README's "Supported Models" section, MLX-VLM provides validated support for the following architectures:

- **Qwen2-VL**: Alibaba's vision-language model (e.g., `mlx-community/Qwen2-VL-2B-Instruct-4bit`)
- **Qwen2.5-VL**: Enhanced successor with improved video understanding capabilities
- **Idefics3**: Hugging Face's multimodal architecture for image and video reasoning
- **LLaVA**: Large Language and Vision Assistant for general vision-language tasks

While additional models such as **Pixtral** and **llava-interleaved** appear in example notebooks, they are not part of the formal supported list and may have limited functionality.

## Loading and Generating With Supported Models

Each supported model follows the same initialization pattern using `mlx_vlm.load()` and `mlx_vlm.generate()`, with `apply_chat_template()` handling prompt formatting. Below are implementation examples for each officially supported architecture.

### Qwen2-VL for Image and Video

Qwen2-VL supports both static images and video sequences. The implementation uses `max_pixels` and `fps` parameters to control video preprocessing:

```python
from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template

model_path = "mlx-community/Qwen2-VL-2B-Instruct-4bit"
model, processor = load(model_path)
config = model.config

# Image generation

image_url = "http://images.cocodataset.org/val2017/000000039769.jpg"
prompt = "Describe this image."
formatted = apply_chat_template(processor, config, prompt, num_images=1)
output = generate(model, processor, formatted, [image_url])
print(output)

# Video generation

video_path = "path/to/video.mp4"
video_prompt = "Summarize what happens in this video."
formatted_vid = apply_chat_template(
    processor, config, video_prompt, num_images=0
)
output_vid = generate(
    model, processor, formatted_vid, video=video_path, max_pixels=(224, 224), fps=1.0
)
print(output_vid)

```

### Qwen2.5-VL for Video Analysis

Qwen2.5-VL extends the architecture with refined temporal understanding, making it ideal for detailed video description tasks:

```python
model_path = "mlx-community/Qwen2.5-VL-32B-Instruct-8bit"
model, processor = load(model_path)
config = model.config

video_path = "path/to/video.mp4"
prompt = "Explain the main events in this clip."

formatted = apply_chat_template(processor, config, prompt)
output = generate(
    model,
    processor,
    formatted,
    video=video_path,
    max_pixels=(224, 224),
    fps=1.0,
)
print(output)

```

### Idefics3 for Video Summarization

Idefics3 provides competitive performance for vision-language tasks with efficient memory usage on Apple Silicon:

```python
model_path = "mlx-community/Idefics3-8B-Instruct-4bit"
model, processor = load(model_path)

video_path = "path/to/video.mp4"
prompt = "Provide a concise summary of the video."

formatted = apply_chat_template(processor, model.config, prompt)
output = generate(
    model,
    processor,
    formatted,
    video=video_path,
    max_pixels=(224, 224),
    fps=1.0,
)
print(output)

```

### LLaVA for Video Description

LLaVA remains a standard for general-purpose vision-language inference, supporting both single images and video sequences:

```python
model_path = "mlx-community/LLaVA-13B-V1.5-4bit"
model, processor = load(model_path)

video_path = "path/to/video.mp4"
prompt = "Describe the scene and any actions happening."

formatted = apply_chat_template(processor, model.config, prompt)
output = generate(
    model,
    processor,
    formatted,
    video=video_path,
    max_pixels=(224, 224),
    fps=1.0,
)
print(output)

```

## Model Validation and Architecture

MLX-VLM determines model capabilities through specific validation logic in the source code. The `is_video_model` function in [`mlx_vlm/video_generate.py`](https://github.com/Blaizzy/mlx-vlm/blob/main/mlx_vlm/video_generate.py) validates whether a loaded architecture supports video inference, while [`mlx_vlm/utils.py`](https://github.com/Blaizzy/mlx-vlm/blob/main/mlx_vlm/utils.py) provides `get_model_and_args` to handle model instantiation and configuration loading.

Abstract base classes defining the interface for these models reside in [`mlx_vlm/models/base.py`](https://github.com/Blaizzy/mlx-vlm/blob/main/mlx_vlm/models/base.py), which establishes the contract for vision, language, and multimodal components. Reference implementations, such as the lightweight SmolVLM example in [`mlx_vlm/models/smolvlm/smolvlm.py`](https://github.com/Blaizzy/mlx-vlm/blob/main/mlx_vlm/models/smolvlm/smolvlm.py), demonstrate how new architectures can extend these base classes to join the supported model roster.

## Summary

- **MLX-VLM** officially supports **Qwen2-VL**, **Qwen2.5-VL**, **Idefics3**, and **LLaVA** for multimodal inference.
- All four models support **video, image, and text inputs** through the unified `generate()` API.
- Model validation occurs via `is_video_model()` in [`video_generate.py`](https://github.com/Blaizzy/mlx-vlm/blob/main/video_generate.py) and configuration handling in [`utils.py`](https://github.com/Blaizzy/mlx-vlm/blob/main/utils.py).
- Experimental models like **Pixtral** exist in examples but lack official support documentation.
- Each model loads via `mlx_vlm.load()` and processes prompts through `apply_chat_template()` before generation.

## Frequently Asked Questions

### What distinguishes Qwen2.5-VL from Qwen2-VL in MLX-VLM?

Both models share the same initialization pattern through `mlx_vlm.load()`, but Qwen2.5-VL represents an updated architecture with enhanced video understanding capabilities. While Qwen2-VL handles both images and videos effectively, Qwen2.5-VL typically offers improved temporal reasoning for complex video analysis tasks according to the repository documentation.

### Are experimental models like Pixtral fully supported?

Pixtral and llava-interleaved appear in example notebooks within the repository, but they are **not** part of the formal "Supported Models" list in the README. These implementations may work for specific use cases but lack the guaranteed compatibility and validation provided for the four official architectures.

### How does MLX-VLM validate video support for a model?

The library checks video compatibility through the `is_video_model` function located in [`mlx_vlm/video_generate.py`](https://github.com/Blaizzy/mlx-vlm/blob/main/mlx_vlm/video_generate.py). This validation occurs before generation, ensuring that the loaded architecture can properly process video inputs using the `max_pixels` and `fps` parameters specified in the `generate()` call.

### Where are the base classes for supported models defined?

Abstract base classes that define the interface for all supported vision-language models reside in [`mlx_vlm/models/base.py`](https://github.com/Blaizzy/mlx-vlm/blob/main/mlx_vlm/models/base.py). These classes establish the contract for vision, language, and multimodal components, while specific implementations like SmolVLM demonstrate the extension pattern in [`mlx_vlm/models/smolvlm/smolvlm.py`](https://github.com/Blaizzy/mlx-vlm/blob/main/mlx_vlm/models/smolvlm/smolvlm.py).