# How to Set a Minimum Speed Requirement for LLM Selection in whichllm

> Control whichllm LLM selection with a minimum speed requirement. Use the --min-speed flag or min_speed argument to set your tokens per second floor and optimize performance.

- Repository: [andy/whichllm](https://github.com/Andyyyy64/whichllm)
- Tags: how-to-guide
- Published: 2026-06-09

---

**Use the `--min-speed` flag in the whichllm CLI, or pass the `min_speed` argument to `rank_models()` in Python, to exclude model variants whose estimated throughput falls below your required tokens-per-second floor.**

whichllm is an open-source LLM recommendation engine that ranks candidate models using hardware-aware benchmarks and performance estimates. If your workload requires a strict **minimum speed requirement for LLM selection**, the tool lets you define a throughput floor in tokens per second so only fast enough models appear in the results.

## How the `--min-speed` Filter Works

The speed filter is implemented across three layers: CLI parsing, core ranking, and performance estimation.

### CLI Entry Point in [`src/whichllm/cli.py`](https://github.com/Andyyyy64/whichllm/blob/main/src/whichllm/cli.py)

When you invoke whichllm from the terminal, the optional `--min-speed` flag is defined by *Typer* in [`src/whichllm/cli.py`](https://github.com/Andyyyy64/whichllm/blob/main/src/whichllm/cli.py). The parsed value is forwarded directly into the ranking pipeline.

### Filtering Logic in [`src/whichllm/engine/ranker.py`](https://github.com/Andyyyy64/whichllm/blob/main/src/whichllm/engine/ranker.py)

The core ranking routine `rank_models` inside [`src/whichllm/engine/ranker.py`](https://github.com/Andyyyy64/whichllm/blob/main/src/whichllm/engine/ranker.py) evaluates every candidate model. For each variant, it calls `estimate_tok_per_sec` to predict throughput. If you supply a `min_speed` value, the routine short-circuits any variant that fails to meet the threshold:

```python
tok_per_sec = estimate_tok_per_sec(...)
if min_speed is not None and tok_per_sec < min_speed:
    continue

```

Only models with an estimated rate greater than or equal to your floor continue through the pipeline and become eligible for final output.

### Speed Estimation in [`src/whichllm/engine/performance.py`](https://github.com/Andyyyy64/whichllm/blob/main/src/whichllm/engine/performance.py)

The actual throughput prediction is produced by `estimate_tok_per_sec`, located in [`src/whichllm/engine/performance.py`](https://github.com/Andyyyy64/whichllm/blob/main/src/whichllm/engine/performance.py). This estimator consumes hardware profiles from [`src/whichllm/hardware/detector.py`](https://github.com/Andyyyy64/whichllm/blob/main/src/whichllm/hardware/detector.py)—which detects your GPU or CPU configuration—to produce a context-aware tokens-per-second value before the filter is applied.

## Usage Examples

You can enforce a **minimum speed requirement for LLM selection** from both the command line and the Python API.

### CLI Examples

Show only the top models that can generate at least 30 tok/s:

```bash
whichllm --min-speed 30

```

Combine the speed floor with other filters, such as a parameter minimum and strict evidence requirements:

```bash
whichllm --min-speed 45 --min-params 7 --evidence strict

```

### Python API Example

For embedded or scripted workflows, import `rank_models` along with the hardware and model fetchers, then pass `min_speed` as a float:

```python
from whichllm.engine.ranker import rank_models
from whichllm.hardware.detector import detect_hardware
from whichllm.models.fetcher import fetch_models

hardware = detect_hardware()
models = fetch_models()

# Enforce a minimum speed requirement for LLM selection of 25 tok/s

results = rank_models(
    models,
    hardware,
    min_speed=25.0,
    top_n=10,
    quant_filter=None,
)
for r in results:
    print(f"{r.model_id}: {r.estimated_tok_per_sec:.1f} tok/s")

```

In this snippet, any variant estimated below 25 tok/s is silently dropped before the final ranked list is returned.

## Summary

- The `--min-speed` CLI flag and `min_speed` Python parameter let you set a throughput floor in **tokens per second**.
- In [`src/whichllm/engine/ranker.py`](https://github.com/Andyyyy64/whichllm/blob/main/src/whichllm/engine/ranker.py), the `rank_models` function skips variants when `tok_per_sec < min_speed`.
- The estimator `estimate_tok_per_sec` in [`src/whichllm/engine/performance.py`](https://github.com/Andyyyy64/whichllm/blob/main/src/whichllm/engine/performance.py) predicts speed using your hardware profile from [`src/whichllm/hardware/detector.py`](https://github.com/Andyyyy64/whichllm/blob/main/src/whichllm/hardware/detector.py).
- This filter composes cleanly with other flags such as `--min-params` and `--evidence`.

## Frequently Asked Questions

### What happens if I do not provide a minimum speed?

If `min_speed` is omitted, the filter is bypassed entirely. As implemented in [`src/whichllm/engine/ranker.py`](https://github.com/Andyyyy64/whichllm/blob/main/src/whichllm/engine/ranker.py), the check `if min_speed is not None and tok_per_sec < min_speed` only executes when you supply an explicit value, so all ranked models are considered regardless of throughput.

### What units does the `--min-speed` parameter use?

The parameter expects a float representing **tokens per second** (tok/s). This aligns with the return value of `estimate_tok_per_sec` in [`src/whichllm/engine/performance.py`](https://github.com/Andyyyy64/whichllm/blob/main/src/whichllm/engine/performance.py), so the comparison is always done in the same unit.

### How does whichllm estimate a model's token generation speed?

Speed estimation is handled by `estimate_tok_per_sec` inside [`src/whichllm/engine/performance.py`](https://github.com/Andyyyy64/whichllm/blob/main/src/whichllm/engine/performance.py), which takes into account the active hardware detected by [`src/whichllm/hardware/detector.py`](https://github.com/Andyyyy64/whichllm/blob/main/src/whichllm/hardware/detector.py). The estimate is hardware-aware, meaning a given model may receive different tok/s predictions on different GPUs or CPUs.

### Can I combine `--min-speed` with other ranking filters?

Yes. The `rank_models` signature accepts multiple constraints simultaneously. In the CLI, flags such as `--min-params` and `--evidence strict` can be used alongside `--min-speed`, and in Python you can pass the corresponding arguments to `rank_models` in [`src/whichllm/engine/ranker.py`](https://github.com/Andyyyy64/whichllm/blob/main/src/whichllm/engine/ranker.py) without conflict.