# How to Set Up and Tune the DSpark Speculative Decoding Confidence Threshold in DS4

> Learn to set up and tune the DSpark speculative decoding confidence threshold using the --dspark-confidence flag. Control token acceptance for faster inference.

- Repository: [Salvatore Sanfilippo/ds4](https://github.com/antirez/ds4)
- Tags: how-to-guide
- Published: 2026-08-09

---

**Set the `--dspark-confidence` flag to a value between 0.0 and 1.0 (default 0.6 for Metal, 0.7 for CUDA) to control how aggressively DSpark accepts draft tokens during speculative decoding.**

DSpark is the speculative-decoding driver in the [antirez/ds4](https://github.com/antirez/ds4) repository, enabled via the `--dspark` flag. When active, DSpark uses a tiny confidence head—a linear projection within the model—to predict whether each draft token is likely correct. The **confidence threshold** determines the cutoff for accepting these predictions, directly balancing inference speed against token accuracy.

## Understanding the Confidence Threshold Mechanism

During a speculative cycle, DSpark generates a draft token sequence followed by a verification pass on the target model. The confidence head outputs a logit for each draft token, which is passed through a sigmoid function and compared against the configured threshold.

### The Verification Process

In [`ds4.c`](https://github.com/antirez/ds4/blob/main/ds4.c), the function `dspark_apply_markov_confidence_lazy_runtime` (line 33472) performs the actual pruning test:

```c
if (sigmoid_stable(confidence_logit) < confidence_threshold) {
    /* reject draft token */
} else {
    /* accept draft token */
}

```

If the sigmoid output falls below the threshold, the draft token is rejected and the verifier re-runs; if above, the token is accepted and the speculative state commits. The threshold value is stored in `engine->dspark_confidence_threshold`, defined at line 36854 in [`ds4.c`](https://github.com/antirez/ds4/blob/main/ds4.c).

### Threshold Impact on Performance

The confidence threshold acts as a dial for speculation aggressiveness:

- **Low values (e.g., 0.4-0.5)**: High acceptance rates, greater speed-up potential, but increased risk of incorrect drafts requiring re-decoding.
- **Default values (0.6-0.7)**: Balanced trade-off between throughput and accuracy.
- **High values (e.g., 0.8-0.9)**: Conservative acceptance, fewer speculative errors, but reduced speed-up due to frequent verifier passes.

## Default Confidence Thresholds by Backend

DS4 applies backend-specific defaults when the `--dspark-confidence` flag is omitted:

| Backend | Default Threshold | Location |
|---------|------------------|----------|
| **Metal** | 0.6 | [`ds4_help.c`](https://github.com/antirez/ds4/blob/main/ds4_help.c) (line 1868) |
| **CUDA / ROCm** | 0.7 | [`ds4_help.c`](https://github.com/antirez/ds4/blob/main/ds4_help.c) (line 1868) |

These values are baked into the CLI help text and initialized in the engine struct at runtime.

## Configuring the Confidence Threshold

You can override the default threshold using the `--dspark-confidence` flag across all DS4 binaries. The flag accepts a floating-point value and must be used in conjunction with `--dspark`.

### Interactive Client (ds4)

```bash

# Use a conservative threshold on Metal

ds4 --model mymodel.gguf --dspark --dspark-confidence 0.85

```

### HTTP Server (ds4-server)

```bash

# Run server with aggressive speculation

ds4-server --model mymodel.gguf --dspark --dspark-confidence 0.55 --port 8080

```

### Coding Assistant (ds4-agent)

```bash

# Balanced setting for agent workflows

ds4-agent --model mymodel.gguf --dspark --dspark-confidence 0.65

```

The threshold can also be specified in configuration files ([`ds4.cfg`](https://github.com/antirez/ds4/blob/main/ds4.cfg)) or via environment variables if your wrapper script forwards them, though the CLI flag provides the most direct control.

## Practical Tuning Strategies

Follow these steps to optimize the DSpark speculative decoding confidence threshold for your specific hardware and model:

1. **Start with the backend default**—0.6 for Metal or 0.7 for CUDA/ROCm—to establish a baseline.
2. **Enable verbose logging** using `--verbose` to monitor per-token confidence values and acceptance decisions in real time.
3. **If you observe frequent verification failures** (indicated by "re-decode" messages in logs), raise the threshold to 0.8 or higher to reduce speculative errors.
4. **For maximum throughput on high-end GPUs**, lower the threshold to 0.5 or 0.55, accepting the occasional re-decoding step in exchange for higher acceptance rates.
5. **Validate model support**—DS4 prints `"confidence: yes"` during model load if the confidence head is present. If the model lacks this head, the threshold parameter is ignored and DSpark falls back to default MTP behavior.

## Implementation Details

The threshold check occurs within the speculative decoding loop. Here is the relevant excerpt from [`ds4.c`](https://github.com/antirez/ds4/blob/main/ds4.c) showing how the engine applies the threshold:

```c
float confidence_threshold = s->engine->dspark_confidence_threshold;
if (sigmoid_stable(confidence_logit) < confidence_threshold) {
    /* Draft token rejected – fall back to verifier */
    reject_token();
} else {
    /* Accept draft token */
    commit_token();
}

```

Unit tests in [`tests/ds4_test.c`](https://github.com/antirez/ds4/blob/main/tests/ds4_test.c) (lines 6298-6299) verify this behavior by setting a custom threshold of 0.9 and asserting correct rejection logic.

## Summary

- The **DSpark confidence threshold** controls draft token acceptance in DS4's speculative decoding pipeline.
- **Default values** are 0.6 for Metal and 0.7 for CUDA/ROCm, stored in `engine->dspark_confidence_threshold`.
- Set the threshold via `--dspark-confidence <float>` in `ds4`, `ds4-server`, or `ds4-agent`.
- **Lower values** increase speed but risk more verification failures; **higher values** improve accuracy at the cost of throughput.
- The model must contain a confidence head (indicated by `"confidence: yes"` at load time) for this parameter to take effect.

## Frequently Asked Questions

### What is the default DSpark confidence threshold for Metal versus CUDA?

The Metal backend defaults to **0.6**, while CUDA and ROCm backends default to **0.7**. These values are defined in the CLI help text within [`ds4_help.c`](https://github.com/antirez/ds4/blob/main/ds4_help.c) and applied to `engine->dspark_confidence_threshold` during engine initialization.

### How do I know if my model supports confidence-based speculative decoding?

During model loading, DS4 prints a diagnostic message indicating `"confidence: yes"` if the model includes a confidence head, or `"confidence: no"` if it does not. If the head is absent, the `--dspark-confidence` flag is ignored and DSpark operates in fallback mode without confidence-based pruning.

### What happens if I set the confidence threshold too low?

Setting the threshold too low (e.g., 0.3 or 0.4) causes the engine to accept draft tokens that are likely incorrect. This increases the **acceptance rate** and speculative speed-up initially, but results in frequent verification failures and re-decoding steps, potentially reducing overall throughput and increasing latency.

### Can I adjust the confidence threshold without restarting the server?

No, the confidence threshold is parsed at startup and stored in the engine struct (`engine->dspark_confidence_threshold` in [`ds4.c`](https://github.com/antirez/ds4/blob/main/ds4.c)). Changes require restarting the `ds4`, `ds4-server`, or `ds4-agent` process with the new `--dspark-confidence` value.