# How to Switch to Target-Only AR Decoding for a Specific Request in MTPLX

> Easily switch to target-only AR decoding for specific MTPLX requests. Set generation_mode to 'ar' in your payload, no server restart needed.

- Repository: [Youssof Altoukhi/MTPLX](https://github.com/youssofal/MTPLX)
- Tags: how-to-guide
- Published: 2026-09-02

---

**Set the `generation_mode` parameter to `"ar"` in your request payload to force MTPLX to use target-only autoregressive decoding for that specific call without restarting the server.**

MTPLX supports both speculative **Multitoken Prediction (MTP)** and standard autoregressive (AR) generation. While MTP uses a draft head to accelerate token generation, you can switch to target-only AR decoding for a specific request—useful for baselines, debugging, or when your model checkpoint lacks an MTP head. This toggle happens at the request level and does not require reloading the runtime.

## Understanding MTPLX Generation Modes

MTPLX operates in two distinct generation modes:

- **MTP (default)** – Uses the draft head for speculative decoding, validating multiple tokens per forward pass.
- **AR-only (target-only)** – Returns only logits from the target model, bypassing the draft head entirely. This mode is triggered when the runtime’s `mtp_enabled` flag is set to `False`.

The mode is determined per-request via the `generation_mode` field in the JSON payload or the `--no-mtp` CLI flag.

## How to Enable Target-Only AR Decoding Per Request

### Using the HTTP API (cURL)

Add `"generation_mode": "ar"` to your JSON body. In [`mtplx/server/openai.py`](https://github.com/youssofal/MTPLX/blob/main/mtplx/server/openai.py) (line 26729), the server extracts this field and stores it in the request metadata.

```bash
curl -X POST https://your-mtplx-host/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
        "model": "gpt-4o-mini",
        "messages": [{"role":"user","content":"Explain quantum tunneling"}],
        "generation_mode": "ar"
      }'

```

The server routes this request through the target-only AR path, returning only logits for each token.

### Using the Python Client

If you use an OpenAI-compatible Python client, pass `generation_mode="ar"` as an extra argument:

```python
import openai

client = openai.OpenAI(base_url="https://your-mtplx-host/v1")
resp = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Explain quantum tunneling"}],
    generation_mode="ar",          # Forces target-only AR for this call

)

print(resp.choices[0].message.content)

```

### Using the CLI

For one-off requests, use the `--no-mtp` flag or the `/mtp` REPL command. In [`mtplx/commands/public.py`](https://github.com/youssofal/MTPLX/blob/main/mtplx/commands/public.py) (lines 720‑728), the CLI normalizes `--no-mtp` to `generation_mode="ar"`.

```bash

# Launch REPL in AR-only mode

mtplx start cli --no-mtp

# Or toggle dynamically inside the REPL

> /mtp off
> ask "Explain quantum tunneling"

```

The `/mtp off` command sets the internal `no_mtp` flag to `True`, which translates to `generation_mode="ar"` for the next request (see [`mtplx/cli.py`](https://github.com/youssofal/MTPLX/blob/main/mtplx/cli.py), lines 304‑313).

## Internal Implementation Details

### Request Parsing in openai.py

When a request arrives, [`mtplx/server/openai.py`](https://github.com/youssofal/MTPLX/blob/main/mtplx/server/openai.py) extracts the `generation_mode` value from the incoming JSON and injects it into the request metadata. This metadata travels with the request through the entire pipeline, ensuring the flag persists for the duration of generation.

### Runtime Dispatch in generation.py

The actual mode switch happens in [`mtplx/generation.py`](https://github.com/youssofal/MTPLX/blob/main/mtplx/generation.py) (line 776) inside the `forward_ar` function. When `runtime.mtp_enabled` is `False`, the function returns `(logits, hidden)` with `hidden=None`, signaling the target-only AR loop to proceed without speculative drafts.

### CLI Normalization in public.py

In [`mtplx/commands/public.py`](https://github.com/youssofal/MTPLX/blob/main/mtplx/commands/public.py) (lines 720‑728), the CLI layer reconciles the `--no-mtp` flag with any explicit `generation_mode` parameter passed via the API. If either is present, the system sets the internal representation to `"ar"` before dispatching to the runtime.

## Mixing AR-Only and MTP Requests

Because the mode is attached to request metadata rather than global state, you can alternate between modes without restarting the server. After processing an AR-only request, re-enable MTP for the next call:

```bash

# AR-only request

> /mtp off
> ask "Explain quantum tunneling"

# Back to MTP speculative decoding

> /mtp on
> ask "Summarize the previous answer"

```

The `/mtp on` command resets the internal flag to `False`, restoring the default MTP pipeline for subsequent requests.

## Summary

- **Target-only AR decoding** is enabled by setting `generation_mode: "ar"` in the request payload.
- The **request metadata** carries this flag through [`mtplx/server/openai.py`](https://github.com/youssofal/MTPLX/blob/main/mtplx/server/openai.py), allowing per-request toggling.
- When `mtp_enabled` is `False`, [`mtplx/generation.py`](https://github.com/youssofal/MTPLX/blob/main/mtplx/generation.py) executes the `forward_ar` path without draft heads.
- You can **mix modes** on the same server instance; the runtime adjusts dynamically without model reloading.
- The **CLI** provides `--no-mtp` and `/mtp off` shortcuts that normalize to the same internal flag.

## Frequently Asked Questions

### Can I mix AR-only and MTP requests on the same server instance?

Yes. The generation mode is stored in per-request metadata parsed by [`mtplx/server/openai.py`](https://github.com/youssofal/MTPLX/blob/main/mtplx/server/openai.py). The server keeps the same loaded runtime and simply toggles the `mtp_enabled` flag for the duration of each request, allowing you to alternate between speculative and target-only decoding without restarting the process.

### Does switching to AR-only mode require reloading the model?

No. The runtime remains resident in memory. When `generation_mode="ar"` is detected, the system temporarily sets `runtime.mtp_enabled = False` for that specific request, causing `forward_ar` in [`mtplx/generation.py`](https://github.com/youssofal/MTPLX/blob/main/mtplx/generation.py) to skip the draft head and return logits only.

### What happens if I use both the `--no-mtp` CLI flag and the `generation_mode="ar"` API parameter?

The CLI handling in [`mtplx/commands/public.py`](https://github.com/youssofal/MTPLX/blob/main/mtplx/commands/public.py) (lines 720‑728) normalizes both inputs to the same internal representation. If you specify `--no-mtp` globally but override with `generation_mode="mtp"` in a specific API call, the explicit API parameter typically takes precedence for that request.

### Is target-only AR faster or slower than MTP?

Target-only AR decoding eliminates the overhead of the draft head and validation logic, but it loses the latency benefits of speculative generation. MTP is generally faster for throughput, while AR-only provides a deterministic baseline useful for benchmarking or when the checkpoint lacks an MTP head.