How to Use the WhichLLM GPU Simulator for Planning: Complete Guide with Code Examples

The WhichLLM GPU simulator lets you emulate any GPU configuration—including VRAM and bandwidth specs—without physical hardware, enabling you to plan which LLMs will run on target machines before purchase or deployment.

Planning hardware for large language model deployment requires knowing exactly which models fit within your GPU memory constraints. The WhichLLM GPU simulator, found in the Andyyyy64/whichllm repository, creates synthetic hardware profiles that the ranking engine treats as real devices. This capability allows you to test deployment scenarios across NVIDIA, AMD, and Apple Silicon architectures using only the CLI or Python API.

How the GPU Simulator Architecture Works

The simulation system creates typed data objects that bypass hardware detection entirely. According to the source code, the architecture consists of four integrated components that transform user input into actionable hardware profiles.

CLI Flag Handling and Validation

The entry point resides in src/whichllm/cli.py, specifically within the _apply_gpu_overrides function (lines 99-113). When you pass --gpu and --vram flags, the CLI validator ensures compatibility before forwarding parameters to the simulator core.

The Synthetic GPU Factory

The create_synthetic_gpu function in src/whichllm/hardware/gpu_simulator.py (lines 93-70) serves as the primary factory. It accepts a GPU name string and optional VRAM override, then queries the dbgpu database for specifications. If the database lacks the specific model, the function falls back to static tables or Apple Silicon-specific logic.

Data Structures for Hardware Profiles

Two typed containers define the simulation output:

• GPUInfo (defined in src/whichllm/hardware/types.py, lines 6-16): Stores vendor, VRAM bytes, memory bandwidth, compute capability, and shared memory flags
• HardwareInfo (same file): Wraps GPU lists for consumption by the ranker

Lookup Helpers for Edge Cases

The simulator includes specialized resolution logic for scenarios where database lookups fail:

• Apple Silicon short-circuit: The _lookup_apple_silicon function (lines 70-90) detects Apple chips and applies unified memory defaults
• Static bandwidth tables: The _lookup_static_bandwidth function (lines 99-104) provides fallback bandwidth values when database entries are missing

Simulation Flow

1. User invokes CLI or Python API with target GPU specifications
2. _apply_gpu_overrides calls create_synthetic_gpu
3. The factory normalizes the GPU name and resolves VRAM/bandwidth via database or fallback tables
4. A GPUInfo instance (e.g., RTX 4090 (simulated)) populates HardwareInfo
5. The ranker consumes hardware.gpus to apply VRAM-based thresholds and return compatible models

Because the simulator only instantiates data objects, it never accesses physical hardware drivers—making it safe to run on any machine regardless of actual GPU presence.

Simulating GPUs via the Command Line

The most common use case involves the --gpu and --vram flags to test deployment scenarios for hardware you do not yet own.

# Simulate an NVIDIA RTX 4090 with 24 GB VRAM

whichllm --gpu "RTX 4090" --vram 24

This command prints a ranked list of LLMs that would run on a machine matching those specifications. The simulator passes the synthetic profile through the same compatibility pipeline used for real hardware detection.

Programmatic GPU Simulation in Python

For integration into planning scripts or automated testing pipelines, import the simulator directly from the hardware module.

Basic Synthetic GPU Creation

from whichllm.hardware.gpu_simulator import create_synthetic_gpu
from whichllm.hardware.types import HardwareInfo

# Build a fake GPU description

gpu = create_synthetic_gpu("RTX 4090", vram_override_gb=24)

# Wrap it in a HardwareInfo object (the CLI does this automatically)

hardware = HardwareInfo(gpus=[gpu])

print(hardware.gpus[0])

# → GPUInfo(name='RTX 4090 (simulated)', vendor='nvidia',

#           vram_bytes=25769803776, memory_bandwidth_gbps=..., shared_memory=False)

Simulating Apple Silicon Chips

Apple Silicon requires special handling because the dbgpu database does not contain these entries. The simulator automatically applies default unified memory sizes and bandwidth values.

gpu = create_synthetic_gpu("M2 Max")   # No VRAM override needed

print(gpu)

# → GPUInfo(name='Apple M2 Max (simulated)', vendor='apple',

#           vram_bytes=34359738368, memory_bandwidth_gbps=100.0, shared_memory=True)

The function automatically assigns 32 GB for M2 Max and marks shared_memory=True to indicate unified memory architecture.

Handling Unknown GPUs with Manual Overrides

When the simulator cannot resolve a GPU name through the database or static tables, it raises a ValueError indicating you must specify VRAM manually.

try:
    gpu = create_synthetic_gpu("MyCustomGPU")
except ValueError as e:
    print(f"Could not find specs: {e}")
    # → "Unknown GPU 'MyCustomGPU'. Use --vram to specify VRAM in GB."

You can still create the simulation by providing the VRAM override:

gpu = create_synthetic_gpu("MyCustomGPU", vram_override_gb=16)

Planning Workflows and What-If Analysis

The GPU simulator enables three critical planning capabilities that reduce hardware procurement risk:

• What-If Scenarios: Test various GPU sizes without physical card swapping. Compare RTX 4090 vs. RTX 3090 configurations instantly to determine if the premium hardware justifies the cost for your specific model requirements.
• Cross-Platform Compatibility: Simulate Apple Silicon GPUs that the underlying dbgpu database cannot detect. Plan M3 Max deployments from a Linux workstation.
• Deterministic Benchmarking: Feed identical synthetic HardwareInfo objects into the ranker to produce reproducible compatibility reports across different host machines, ensuring consistent planning data for team collaboration.

Because the ranker instantaneously prunes models exceeding the synthetic VRAM limits, you receive immediate feedback on which quantization levels (4-bit, 8-bit) and context lengths will fit your target hardware.

Summary

• The WhichLLM GPU simulator creates synthetic GPUInfo objects through create_synthetic_gpu in src/whichllm/hardware/gpu_simulator.py
• Use CLI flags --gpu and --vram for quick planning queries, or invoke the Python API for automated hardware compatibility testing
• The simulator handles Apple Silicon via _lookup_apple_silicon and unknown GPUs via manual VRAM overrides
• Synthetic hardware profiles flow into HardwareInfo objects that the ranker processes identically to real detected hardware
• Zero hardware access occurs during simulation, making it safe to run on any machine regardless of actual GPU presence

Frequently Asked Questions

How does the WhichLLM GPU simulator handle GPUs not in the database?

When create_synthetic_gpu encounters an unknown GPU name, it first attempts fuzzy matching against static tables in src/whichllm/data/gpu.py. If no match exists, it raises a ValueError prompting you to use the --vram flag or vram_override_gb parameter. You can still simulate the hardware by manually specifying the VRAM in gigabytes, though bandwidth estimates may use conservative defaults.

Can I simulate Apple Silicon GPUs on a Linux or Windows machine?

Yes. The simulator detects Apple Silicon chip names (M1, M2, M3 series) through the _lookup_apple_silicon helper and applies predefined unified memory sizes and bandwidth values. This works on any host operating system because the function only creates data objects without calling platform-specific hardware APIs.

What is the difference between HardwareInfo and GPUInfo?

GPUInfo (defined in src/whichllm/hardware/types.py) is a typed dataclass storing individual GPU attributes: name, vendor, VRAM bytes, bandwidth, and shared memory status. HardwareInfo is a higher-level container that holds a list of GPUInfo objects along with system-wide parameters. The ranker consumes HardwareInfo to determine model compatibility across single or multi-GPU configurations.

Does the simulator affect actual hardware performance or drivers?

No. The simulator purely instantiates data structures in memory. It never loads GPU drivers, allocates VRAM, or executes CUDA/OpenCL kernels. You can safely simulate a 48GB A6000 on a laptop with integrated graphics—the tool only calculates which models would fit based on the synthetic specifications you provide.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}