# Load an image (local file, URL, or base64)

> Learn how to load images from local files URLs or base64 into MLX VLM. Understand the generation pipeline for efficient image processing and model integration.

- Repository: [Prince Canuma/mlx-vlm](https://github.com/Blaizzy/mlx-vlm)
- Tags: how-to-guide
- Published: 2026-04-05

---

` token followed by a newline (lines 462–474) to terminate the thinking phase and proceed to the final answer.

## Stage 3: Text Generation and KV-Cache Management

The core generation logic resides in [`mlx_vlm/generate.py`](https://github.com/Blaizzy/mlx-vlm/blob/main/mlx_vlm/generate.py), which implements memory-efficient token streaming with sophisticated cache handling.

### KV-Cache Initialization

Before generation begins, the pipeline creates a list of per-layer `KVCache` objects via `make_prompt_cache()` (lines 267–271):

```python
kwargs["prompt_cache"] = cache.make_prompt_cache(
    model.language_model,
    max_kv_size=kwargs.get("max_kv_size", None),
)

```

The `PromptCacheState` class (lines 334–352) stores these key-value pairs for prompt reuse across conversation turns, enabling efficient multi-turn dialogue.

### Chunked Prefill for Large Contexts

For prompts exceeding `prefill_step_size` (default 2048 tokens), `generate_step()` performs **chunked prefill** (lines 332–345):

```python
while inputs_embeds.shape[1] > prefill_step_size:
    model.language_model(inputs[:n_to_process], inputs_embeds[:n_to_process], cache=prompt_cache, ...)
    mx.eval([c.state for c in prompt_cache])

```

This chunking strategy reduces peak memory usage during the initial prompt processing phase while maintaining computational efficiency.

### Streaming Token Generation

The `_step()` function drives the token-by-token generation loop (lines 351–354 and 566–571):

```python
y, logprobs = _step(input_ids, inputs_embeds=inputs_embeds)
while True:
    next_y, next_logprobs = _step(y[None])
    yield y.item(), logprobs

```

Within each `_step()` invocation:
1. The model computes logits for the last token position
2. **Logits processors** apply repetition penalties and top-p/k sampling (lines 555–564)
3. Optional **KV-cache quantization** occurs via `maybe_quantize_kv_cache()` (lines 442–470)
4. The `sampler` selects the next token (lines 472–477)
5. The loop yields `GenerationResult` objects containing generated text, timing statistics, and memory metrics (lines 689–735)

### Cache-State Reuse for Conversational AI

When provided with a `prompt_cache_state`, the pipeline computes the longest common prefix between the new prompt and cached tokens using `PromptCacheState.find_prefix_length()` (lines 358–366). Only tokens beyond this common prefix undergo prefill, drastically accelerating multi-turn chat applications (lines 672–689).

After generation completes, the detokenizer finalizes text segments, the KV-cache saves back to `PromptCacheState` (lines 779–784), and `mx.clear_cache()` frees temporary buffers (line 786).

## Practical Code Examples

### Example 1: Single Image Captioning

```python
from mlx_vlm.utils import load, load_image
from mlx_vlm.generate import generate

model, processor = load("mlx-community/Qwen2.5-VL-7B-Instruct-4bit")

# Load an image (local file, URL, or base64)

image = load_image("https://example.com/cat.jpg")

# Build the chat message list

messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "image": image},
            {"type": "text", "text": "Describe this picture."},
        ],
    }
]

# Convert to the processor-friendly format

text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)

# Run generation (default max-tokens=100, temperature=0.7)

result = generate(
    model,
    processor,
    prompt=text,
    image=image,
    verbose=True,
)

print("Caption:", result.text)

```

> **Source reference:** [`video_generate.py`](https://github.com/Blaizzy/mlx-vlm/blob/main/video_generate.py) lines 25-35 demonstrate message construction and vision extraction.

### Example 2: Video Description with Frame Sampling

```python
from mlx_vlm.utils import load
from mlx_vlm.video_generate import main as video_cli

# The CLI wrapper parses arguments and runs the pipeline

args = [
    "--video", "sample.mp4",
    "--prompt", "What is happening in this clip?",
    "--fps", "2",                # sample 2 frames per second

    "--max-frames", "20",        # limit to 20 frames

    "--temperature", "0.0",
    "--model", "mlx-community/Qwen2.5-VL-7B-Instruct-4bit",
]

video_cli(args)

```

> **Source reference:** [`video_generate.py`](https://github.com/Blaizzy/mlx-vlm/blob/main/video_generate.py) lines 38-50 handle smart frame selection; lines 61-86 manage video preprocessing.

### Example 3: Multi-Turn Chat with KV-Cache Reuse

```python
from mlx_vlm.utils import load
from mlx_vlm.generate import stream_generate, PromptCacheState

model, processor = load("mlx-community/Qwen2.5-VL-7B-Instruct-4bit")
state = PromptCacheState()        # persists KV-cache across turns

def ask(prompt):
    # Stream generation for interactive responses

    for result in stream_generate(
        model,
        processor,
        prompt,
        prompt_cache_state=state,   # reuse cached computations

        max_tokens=64,
    ):
        print(result.text, end="", flush=True)
    print()

ask("What is the capital of France?")
ask("Now give me a short history of that city.")   # only new tokens are processed

```

> **Source reference:** [`generate.py`](https://github.com/Blaizzy/mlx-vlm/blob/main/generate.py) lines 668-689 implement cache prefix detection; `PromptCacheState` is defined in lines 334-352.

## Summary

- **The MLX-VLM generation pipeline** orchestrates model loading, multimodal preprocessing, and streaming text generation optimized for Apple silicon hardware.
- **Model initialization** in [`mlx_vlm/utils.py`](https://github.com/Blaizzy/mlx-vlm/blob/main/mlx_vlm/utils.py) handles quantized weight loading, processor configuration, and stopping criteria setup via `load()` (lines 63–98).
- **Multimodal preparation** detects vision inputs through `process_vision_info()` in [`mlx_vlm/video_generate.py`](https://github.com/Blaizzy/mlx-vlm/blob/main/mlx_vlm/video_generate.py), applies smart resizing, and converts media to tensors via `process_inputs()` (lines 18–30).
- **Generation** utilizes chunked prefill (lines 332–345) and KV-cache reuse (lines 672–689) to minimize memory usage and accelerate multi-turn conversations.
- **Streaming output** yields tokens through `_step()` with optional quantization and thinking-budget constraints for real-time inference.

## Frequently Asked Questions

### How does MLX-VLM handle video inputs for models without native video support?

When `is_video_model()` returns false (lines 14–18 in [`video_generate.py`](https://github.com/Blaizzy/mlx-vlm/blob/main/video_generate.py)), the pipeline triggers `VideoFrameExtractor` to sample discrete frames from the video. These frames are processed as a list of individual images rather than unified video tensors, allowing any image-capable VLM to analyze video content through frame-by-frame processing.

### What is the purpose of the PromptCacheState in MLX-VLM generation?

`PromptCacheState` (lines 334–352 in [`generate.py`](https://github.com/Blaizzy/mlx-vlm/blob/main/generate.py)) maintains key-value caches across conversation turns to avoid recomputing attention states for previously seen tokens. By computing the longest common prefix between the current prompt and cached history via `find_prefix_length()` (lines 358–366), the pipeline only processes new tokens during prefill, significantly reducing latency in multi-turn chat applications.

### How does the chunked prefill mechanism work in MLX-VLM?

When prompts exceed `prefill_step_size` (default 2048 tokens), the `generate_step()` function (lines 332–345) splits the prompt into chunks and processes them sequentially through the language model with `mx.eval()` synchronization between chunks. This prevents memory spikes during large context processing while maintaining computational efficiency on MLX's unified memory architecture.

### What are thinking tokens and how does the pipeline manage them?

Thinking tokens represent intermediate reasoning steps generated by the model. When `--enable-thinking` is enabled, `ThinkingBudgetCriteria` (lines 440–515 in [`utils.py`](https://github.com/Blaizzy/mlx-vlm/blob/main/utils.py)) monitors the generation stream and enforces a `thinking_budget` limit. If reasoning exceeds the budget, the pipeline forces a `</think>` token followed by a newline (lines 462–474) to terminate the thinking phase and proceed to the final answer.