How to Compare GPU Upgrade Options with WhichLLM: A Complete Guide

The whichllm upgrade command simulates different graphics cards against your current hardware to show exactly how each GPU would change your language model recommendations, quality scores, and inference speed.

WhichLLM is an open-source tool by Andyyyy64 that helps you find the optimal language models for your specific hardware. When you want to compare GPU upgrade options with WhichLLM, the built-in upgrade command lets you simulate target graphics cards without physically installing them, showing side-by-side comparisons of quality scores and token throughput. This feature is implemented in the src/whichllm/cli.py entry point and documented in docs/cli.md.

Running the Upgrade Comparison from the CLI

Basic Syntax

The whichllm upgrade command accepts one or more GPU names as positional arguments. The tool first builds a HardwareInfo object describing your current CPU, RAM, OS, and detected GPUs according to the definition in src/whichllm/hardware/types.py (lines 18-27). It then runs the ranking pipeline for each specified target GPU.

# Compare two NVIDIA cards against your current setup

whichllm upgrade RTX-4090 RTX-5090

# Compare against Apple Silicon

whichllm upgrade "Apple M4 Max"

Overriding VRAM Detection

Sometimes drivers report incorrect VRAM amounts. Use the --vram flag to manually specify the memory size in gigabytes, which directly modifies the vram_bytes field in the GPUInfo object before simulation.

whichllm upgrade RTX-3060 --vram 12

JSON Output for Automation

Add --json to get machine-readable output for integration with external scripts or CI/CD pipelines.

whichllm upgrade RTX-4090 RTX-5090 --json

How the Comparison Works Under the Hood

Step 1: Capturing Your Current Hardware

In src/whichllm/hardware/types.py, the HardwareInfo class defines the data structure that captures your CPU name, core count, RAM, OS, and detected GPUs. This serves as the baseline for comparison.

Step 2: Establishing the Baseline Ranking

The tool runs the standard ranking pipeline against your current HardwareInfo to determine the top-recommended model, optimal quantization level, quality score, and estimated tokens-per-second. These results are stored in current_results.

Step 3: Simulating Target GPUs

For each GPU you specify on the command line, WhichLLM creates a synthetic HardwareInfo clone that preserves your current CPU, RAM, and OS but replaces the GPU list with a single GPUInfo entry matching the requested name. If you use the --vram flag, the tool overrides the detected VRAM value in the synthetic GPUInfo.

Step 4: Summarizing Results with _summarize_row

The helper function _summarize_row in src/whichllm/output/display.py (lines 37-59) extracts key metrics from each ranking: GPU name, VRAM, top model ID, quantization level, quality score, and token throughput.

Step 5: Computing Deltas and Verdicts

For each target GPU, the code in display_upgrade computes:

• ΔQ = target_quality - current_quality
• Δtok/s = target_tok_per_sec - current_tok_per_sec

The _upgrade_verdict function in src/whichllm/output/display.py (lines 79-90) then classifies the upgrade value based on these thresholds:

if delta_q >= 12 and delta_speed >= 10:   # ≥12-pt quality lift and ≥10 tok/s

    return "[bold green]worth it[/]"
if delta_q >= 8 or delta_speed >= 20:
    return "[green]meaningful[/]"
if delta_q >= 3 or delta_speed >= 5:
    return "[yellow]marginal[/]"
if delta_q <= -3 or delta_speed <= -5:
    return "[red]downgrade[/]"
return "[dim]flat[/]"

Step 6: Rendering the Comparison Table

The display_upgrade function (lines 92-147 in src/whichllm/output/display.py) builds a Rich Table with columns for the setup name, GPU model, VRAM, best model ID, quantization, quality score, tokens/second, the calculated deltas, and the final verdict.

Using the Comparison in Python Scripts

For custom tooling or automation, you can reproduce the upgrade logic programmatically using the internal APIs:

from whichllm.hardware.types import HardwareInfo, GPUInfo
from whichllm.output.display import display_upgrade, _summarize_row

# current hardware (normally discovered automatically)

current_hw = HardwareInfo(...)
current_results = rank_models(current_hw)   # internal ranking call

# simulate a target GPU

target_hw = HardwareInfo(
    gpus=[GPUInfo(name="RTX 5090", vendor="nvidia", vram_bytes=32 * 1024**3, shared_memory=False)],
    cpu_name=current_hw.cpu_name,
    cpu_cores=current_hw.cpu_cores,
    ram_bytes=current_hw.ram_bytes,
    os=current_hw.os,
)

target_results = rank_models(target_hw)
display_upgrade(current_hw, current_results,
               [("RTX‑5090", target_hw, target_results)])

Summary

• Use whichllm upgrade <GPU_NAMES> to compare GPU upgrade options with WhichLLM without physical hardware changes.
• The tool simulates target GPUs by creating synthetic HardwareInfo objects while preserving your actual CPU and RAM configuration.
• Results show quality deltas (ΔQ), speed deltas (Δtok/s), and a verdict based on thresholds defined in _upgrade_verdict.
• Override auto-detected VRAM with --vram when your drivers report incorrect values.
• Export results as JSON using --json for integration with external tools.

Frequently Asked Questions

Can I compare multiple GPUs at once?

Yes. Pass multiple GPU names as space-separated arguments to the upgrade command. WhichLLM will simulate each one against your current hardware baseline and display a comparison table showing all targets with their respective deltas and verdicts.

How does WhichLLM handle VRAM detection errors?

If your GPU driver reports incorrect VRAM values, use the --vram flag followed by the correct size in gigabytes. This overrides the auto-detected value in the GPUInfo object before running the simulation, ensuring accurate model recommendations for cards with non-standard memory configurations.

What do the verdict categories mean?

The verdict classification in _upgrade_verdict uses these thresholds: "worth it" requires ≥12 quality points and ≥10 tok/s improvement; "meaningful" requires ≥8 quality points or ≥20 tok/s; "marginal" requires ≥3 quality points or ≥5 tok/s; "downgrade" appears if you lose ≥3 quality points or ≥5 tok/s; otherwise it is "flat".

Can I use this for non-NVIDIA GPUs?

Yes. WhichLLM supports various vendors including Apple Silicon. Pass the GPU name as a quoted string, such as whichllm upgrade "Apple M4 Max", and the tool will simulate against that hardware profile using the same HardwareInfo and GPUInfo data structures.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}