How whichllm Simulates GPUs for Testing LLM Performance

whichllm simulates GPUs by constructing synthetic GPUInfo objects through the create_synthetic_gpu() function in src/whichllm/hardware/gpu_simulator.py, enabling users to evaluate LLM compatibility and speed estimates using the --gpu and --vram flags without physical hardware.

The whichllm open-source tool (implemented in Andyyyy64/whichllm) helps developers determine which Large Language Models can run on their machines. To simulate GPUs for testing LLM performance without owning specific graphics cards, the project implements a sophisticated simulation pipeline that constructs synthetic hardware profiles from a database of over 2000 GPUs.

The GPU Simulation Architecture

Database Lookup and Normalization

The simulation pipeline lives in src/whichllm/hardware/gpu_simulator.py. The _lookup_dbgpu() function queries the dbgpu package—a comprehensive TechPowerUp database containing VRAM capacity, memory bandwidth, and compute capability specifications. The implementation normalizes GPU names and attempts exact, prefixed, substring, and fuzzy matches against GPUDatabase.default() to identify hardware specifications accurately.

Apple Silicon and AMD APU Special Handling

Because the dbgpu database does not include Apple GPUs, the simulator checks a hard-coded map of Apple chip names before querying the database. When matched, it returns synthetic specs with the chip's default unified memory and preset bandwidth values. For AMD hardware, the code detects shared-memory APU configurations by recognizing specific model name patterns, ensuring accurate memory modeling for integrated graphics scenarios.

VRAM and Bandwidth Override Logic

When users provide the --vram flag, the specified value overrides any database entry, allowing custom memory configurations for unreleased or modified hardware. If the database lacks bandwidth data for a specific GPU, the system falls back to a static GPU_BANDWIDTH lookup table. Vendor determination derives from the database's manufacturer field or the Apple/APU special-case logic, ensuring correct identification across hardware generations.

Constructing Synthetic GPU Objects

The create_synthetic_gpu() function assembles a complete GPUInfo dataclass (defined in src/whichllm/hardware/types.py) with the "(simulated)" suffix appended to the device name. This synthetic object includes discovered attributes such as vram_bytes, compute_capability, memory_bandwidth_gbps, and shared_memory flags. The resulting GPUInfo injects into the HardwareInfo object passed to the ranker, enabling the same fit-type checks (full_gpu, partial_offload, cpu_only) used for real hardware detection.

Command-Line and Programmatic Usage

Simulate hardware directly from the terminal to test model compatibility:

# Simulate an NVIDIA RTX 4090 using automatic database lookup

whichllm --gpu "RTX 4090"

# Simulate Apple Silicon with automatic memory detection

whichllm hardware --gpu "Apple M3 Max"

# Override VRAM when the GPU name is ambiguous or unreleased

whichllm --gpu "RTX 5060 Ti" --vram 16

For testing or integration into Python applications:

from whichllm.hardware.gpu_simulator import create_synthetic_gpu

gpu = create_synthetic_gpu("RTX 4090")
print(gpu)

# Output: GPUInfo(name='RTX 4090 (simulated)', vendor='nvidia',

#         vram_bytes=24_000_000_000, compute_capability=(8, 6),

#         memory_bandwidth_gbps=616.0, shared_memory=False)

Summary

• Synthetic GPU Creation: The create_synthetic_gpu() function in src/whichllm/hardware/gpu_simulator.py generates GPUInfo objects that mimic real hardware specifications.
• Database Integration: The dbgpu package provides specifications for over 2000 GPUs, with fallback logic for Apple Silicon and AMD APUs.
• VRAM Override: The --vram flag allows manual memory specification when database entries are missing or insufficient.
• Fit Type Evaluation: Simulated GPUs undergo the same compatibility checks (full_gpu, partial_offload, cpu_only) as physical hardware detected on the system.
• No Hardware Required: Users can explore model downloadability, memory constraints, and speed estimates for hardware they do not physically possess.

Frequently Asked Questions

Can whichllm simulate GPUs not yet released to the market?

Yes, by using the --vram flag to manually specify memory capacity when the GPU name is not present in the dbgpu database. This allows testing against rumored or future hardware specifications, though bandwidth and compute capability estimates may require manual verification since the database lacks official entries.

How does the simulator handle Apple Silicon GPUs differently than NVIDIA or AMD?

The code first checks a hard-coded map of Apple chip names before querying the database, since dbgpu does not include Apple GPUs. This shortcut returns synthetic specs with the chip's default unified memory configuration and preset bandwidth values, ensuring accurate modeling of macOS-based LLM deployment scenarios where unified memory architecture differs from discrete GPUs.

What happens if the GPU name is ambiguous or not found in the database?

The _lookup_dbgpu() function implements a tiered matching strategy attempting exact, prefixed, substring, and fuzzy matches against the database. If no match is found and no --vram override is provided, the system raises a clear error with suggestions to help identify the correct hardware name from the available database entries.

Is the simulated GPU information distinguishable from real hardware detection?

Yes, synthetic GPUInfo objects explicitly include the "(simulated)" suffix in their name field, making it clear when evaluating results that the hardware profile was user-specified rather than auto-detected from the local system. This distinction prevents confusion between actual system capabilities and hypothetical testing scenarios.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}