# How to Use MTP with an MTPLX Model via Command Line

> Learn to use MTP with an MTPLX model via command line. Leverage Multi-Token Prediction for enhanced text generation with the youssofal/MTPLX repository. Disable MTP easily with a flag.

- Repository: [Youssof Altoukhi/MTPLX](https://github.com/youssofal/MTPLX)
- Tags: how-to-guide
- Published: 2026-09-02

---

**MTPLX enables Multi-Token Prediction (MTP) by default for compatible models, requiring only the `--no-mtp` flag to disable it when pure autoregressive generation is needed.**

MTPLX is an open-source inference engine that ships with native speculative decoding capabilities for models featuring an MTP head, such as Qwen 3.5/3.6/3.8 and Gemma 4. The command line interface provides granular control over how to use MTP with an MTPLX model via command line through simple flags, runtime toggles, and advanced configuration contracts.

## Understanding MTP in MTPLX

Multi-Token Prediction (MTP) acts as a speculative decoding mechanism that predicts multiple future tokens in parallel, delivering 1.5–2× speedups while preserving exact sampling semantics. According to the youssofal/MTPLX source code, the behavior is governed by an immutable `MTPContract` dataclass defined in [`mtplx/mtp_patch.py`](https://github.com/youssofal/MTPLX/blob/main/mtplx/mtp_patch.py) (lines 48–62). This contract encodes critical parameters including quantization bits, hidden variants, and pre-quantized module policies that determine how the MTP head operates during inference.

## CLI Flags for MTP Control

### Default Behavior and Basic Toggling

The CLI enables MTP automatically when loading models that support it. In [`mtplx/cli.py`](https://github.com/youssofal/MTPLX/blob/main/mtplx/cli.py), the function `_add_mtp_toggle_args` (lines 77–89) wires the `--mtp` and `--no-mtp` flags into every public command. The following commands accept these flags:

- `mtplx start`
- `mtplx run`
- `mtplx serve`
- `mtplx quickstart`

When you omit any flag, the system defaults to MTP enabled (`--mtp`). To force pure autoregressive generation—useful for debugging or when using models without an MTP head—you must explicitly pass `--no-mtp`.

### Runtime Toggle Commands

For interactive chat sessions started with `mtplx start`, you can switch MTP on or off without restarting the runtime. At the chat prompt, use:

- `/mtp off` — Disables MTP for the next turn only
- `/mtp on` — Re-enables MTP for subsequent turns

This allows dynamic A/B testing between speculative and autoregressive generation during a single session.

## Configuring the MTP Contract

### The MTPContract Dataclass

The runtime contract that determines exact MTP behavior—including hidden variants and quantization policies—is defined in [`mtplx/mtp_patch.py`](https://github.com/youssofal/MTPLX/blob/main/mtplx/mtp_patch.py). The default contract applies automatically, but advanced users can override specific fields such as `mtp_quant_bits` or `mtp_quant_policy` through configuration layers.

### Customizing via User Config

Create or edit `~/.mtplx/config.toml` to specify a custom contract under the `[mtplx]` section:

```toml
[mtplx]
mtplx_mtp_contract = { mtp_quant_bits = 4, mtp_quant_policy = "cyankiwi" }

```

When present, this configuration overrides the default values defined in the source code.

### Environment Variable Override

For temporary overrides or containerized deployments, set the `MTPLX_MTP_CONTRACT` environment variable. The runtime in [`mtplx/commands/public.py`](https://github.com/youssofal/MTPLX/blob/main/mtplx/commands/public.py) reads this variable during initialization, parsing it as JSON to inject custom contract values before the model loads.

## Checking Model Compatibility

Before launching a session, verify that your model supports speculative decoding using the inspection tool:

```bash
mtplx inspect <model-id>

```

This command invokes `inspect_model` in [`mtplx/artifacts.py`](https://github.com/youssofal/MTPLX/blob/main/mtplx/artifacts.py) to print a compatibility report, including a boolean "MTP-compatible" status that indicates whether the checkpoint includes the necessary MTP head for speculative generation.

## Practical Command Examples

Start an interactive chat with default MTP enabled:

```bash
mtplx start --model Youssofal/Qwen3.8-27B-MTPLX-Optimized-Speed

```

Execute a one-shot generation while explicitly ensuring MTP is active:

```bash
mtplx run "Explain how MTP speeds up generation" \
    --model Youssofal/Qwen3.8-27B-MTPLX-Optimized-Speed \
    --mtp

```

Force pure autoregressive generation for a model without an MTP head:

```bash
mtplx start --model mlx-community/Laguna-S-2.1-oQ4e --no-mtp

```

Inspect a model's speculative decoding capabilities before loading:

```bash
mtplx inspect Youssofal/Qwen3.8-27B-MTPLX-Optimized-Speed

```

## Summary

- **MTP is enabled by default** in MTPLX for compatible models; no flags are required to activate it.
- Use **`--no-mtp`** to force pure autoregressive generation when debugging or using incompatible models.
- Toggle speculative decoding dynamically using **`/mtp on`** and **`/mtp off`** during interactive sessions.
- Customize quantization and behavior via the **`mtplx_mtp_contract`** in `~/.mtplx/config.toml` or the **`MTPLX_MTP_CONTRACT`** environment variable.
- Verify compatibility beforehand using **`mtplx inspect`** to check for MTP head presence.

## Frequently Asked Questions

### How do I know if my model supports MTP?

Run `mtplx inspect <model-id>` to generate a compatibility report. The output includes an "MTP-compatible" field that indicates whether the checkpoint contains the necessary head for speculative decoding, as determined by the `inspect_model` function in [`mtplx/artifacts.py`](https://github.com/youssofal/MTPLX/blob/main/mtplx/artifacts.py).

### Can I switch MTP on and off without restarting the CLI?

Yes. During an interactive session started with `mtplx start`, type `/mtp off` to disable speculative decoding for the next turn, or `/mtp on` to re-enable it. These commands take effect immediately without reloading the model weights.

### What is the difference between `--mtp` and `--no-mtp`?

The `--mtp` flag explicitly confirms the default behavior (enabling speculative decoding), while `--no-mtp` disables the MTP head entirely, forcing pure autoregressive generation. The flags are processed in [`mtplx/cli.py`](https://github.com/youssofal/MTPLX/blob/main/mtplx/cli.py) by the `_add_mtp_toggle_args` function and apply to all model-loading commands.

### How do I customize MTP quantization settings?

Define a custom contract in `~/.mtplx/config.toml` under the `[mtplx]` section using the `mtplx_mtp_contract` key, or set the `MTPLX_MTP_CONTRACT` environment variable. These values override the default `MTPContract` dataclass in [`mtplx/mtp_patch.py`](https://github.com/youssofal/MTPLX/blob/main/mtplx/mtp_patch.py) to specify custom quantization bits, policies, or hidden variants.