How to Use MTP with an MTPLX Model via Command Line

MTPLX enables Multi-Token Prediction (MTP) by default for compatible models, requiring only the --no-mtp flag to disable it when pure autoregressive generation is needed.

MTPLX is an open-source inference engine that ships with native speculative decoding capabilities for models featuring an MTP head, such as Qwen 3.5/3.6/3.8 and Gemma 4. The command line interface provides granular control over how to use MTP with an MTPLX model via command line through simple flags, runtime toggles, and advanced configuration contracts.

Understanding MTP in MTPLX

Multi-Token Prediction (MTP) acts as a speculative decoding mechanism that predicts multiple future tokens in parallel, delivering 1.5–2× speedups while preserving exact sampling semantics. According to the youssofal/MTPLX source code, the behavior is governed by an immutable MTPContract dataclass defined in mtplx/mtp_patch.py (lines 48–62). This contract encodes critical parameters including quantization bits, hidden variants, and pre-quantized module policies that determine how the MTP head operates during inference.

CLI Flags for MTP Control

Default Behavior and Basic Toggling

The CLI enables MTP automatically when loading models that support it. In mtplx/cli.py, the function _add_mtp_toggle_args (lines 77–89) wires the --mtp and --no-mtp flags into every public command. The following commands accept these flags:

• mtplx start
• mtplx run
• mtplx serve
• mtplx quickstart

When you omit any flag, the system defaults to MTP enabled (--mtp). To force pure autoregressive generation—useful for debugging or when using models without an MTP head—you must explicitly pass --no-mtp.

Runtime Toggle Commands

For interactive chat sessions started with mtplx start, you can switch MTP on or off without restarting the runtime. At the chat prompt, use:

• /mtp off — Disables MTP for the next turn only
• /mtp on — Re-enables MTP for subsequent turns

This allows dynamic A/B testing between speculative and autoregressive generation during a single session.

Configuring the MTP Contract

The MTPContract Dataclass

The runtime contract that determines exact MTP behavior—including hidden variants and quantization policies—is defined in mtplx/mtp_patch.py. The default contract applies automatically, but advanced users can override specific fields such as mtp_quant_bits or mtp_quant_policy through configuration layers.

Customizing via User Config

Create or edit ~/.mtplx/config.toml to specify a custom contract under the [mtplx] section:

[mtplx]
mtplx_mtp_contract = { mtp_quant_bits = 4, mtp_quant_policy = "cyankiwi" }

When present, this configuration overrides the default values defined in the source code.

Environment Variable Override

For temporary overrides or containerized deployments, set the MTPLX_MTP_CONTRACT environment variable. The runtime in mtplx/commands/public.py reads this variable during initialization, parsing it as JSON to inject custom contract values before the model loads.

Checking Model Compatibility

Before launching a session, verify that your model supports speculative decoding using the inspection tool:

mtplx inspect <model-id>

This command invokes inspect_model in mtplx/artifacts.py to print a compatibility report, including a boolean "MTP-compatible" status that indicates whether the checkpoint includes the necessary MTP head for speculative generation.

Practical Command Examples

Start an interactive chat with default MTP enabled:

mtplx start --model Youssofal/Qwen3.8-27B-MTPLX-Optimized-Speed

Execute a one-shot generation while explicitly ensuring MTP is active:

mtplx run "Explain how MTP speeds up generation" \
    --model Youssofal/Qwen3.8-27B-MTPLX-Optimized-Speed \
    --mtp

Force pure autoregressive generation for a model without an MTP head:

mtplx start --model mlx-community/Laguna-S-2.1-oQ4e --no-mtp

Inspect a model's speculative decoding capabilities before loading:

mtplx inspect Youssofal/Qwen3.8-27B-MTPLX-Optimized-Speed

Summary

• MTP is enabled by default in MTPLX for compatible models; no flags are required to activate it.
• Use --no-mtp to force pure autoregressive generation when debugging or using incompatible models.
• Toggle speculative decoding dynamically using /mtp on and /mtp off during interactive sessions.
• Customize quantization and behavior via the mtplx_mtp_contract in ~/.mtplx/config.toml or the MTPLX_MTP_CONTRACT environment variable.
• Verify compatibility beforehand using mtplx inspect to check for MTP head presence.

Frequently Asked Questions

How do I know if my model supports MTP?

Run mtplx inspect <model-id> to generate a compatibility report. The output includes an "MTP-compatible" field that indicates whether the checkpoint contains the necessary head for speculative decoding, as determined by the inspect_model function in mtplx/artifacts.py.

Can I switch MTP on and off without restarting the CLI?

Yes. During an interactive session started with mtplx start, type /mtp off to disable speculative decoding for the next turn, or /mtp on to re-enable it. These commands take effect immediately without reloading the model weights.

What is the difference between --mtp and --no-mtp?

The --mtp flag explicitly confirms the default behavior (enabling speculative decoding), while --no-mtp disables the MTP head entirely, forcing pure autoregressive generation. The flags are processed in mtplx/cli.py by the _add_mtp_toggle_args function and apply to all model-loading commands.

How do I customize MTP quantization settings?

Define a custom contract in ~/.mtplx/config.toml under the [mtplx] section using the mtplx_mtp_contract key, or set the MTPLX_MTP_CONTRACT environment variable. These values override the default MTPContract dataclass in mtplx/mtp_patch.py to specify custom quantization bits, policies, or hidden variants.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}