What CPU Information Does WhichLLM Gather? A Technical Deep Dive

WhichLLM automatically detects the CPU model name, physical core count, and AVX2/AVX-512 instruction set support to determine which LLM variants can run on a given machine.

WhichLLM is an open-source hardware compatibility tool that helps users identify runnable large language models for their specific machines. To make accurate recommendations, the application must first understand the underlying CPU capabilities, gathering detailed processor information through platform-specific system calls and file parsing.

CPU Model Name Detection

WhichLLM identifies the processor brand and model through detect_cpu_name() in src/whichllm/hardware/cpu.py (lines 100-131). The implementation uses a platform-specific cascade to ensure accurate identification across diverse hardware.

Linux Detection Strategy

On Linux systems, the function first attempts to read /proc/cpuinfo for the "model name" field. If this information is unavailable—common on ARM or aarch64 architectures—it falls back to _cpu_name_from_lscpu to extract the identifier from lscpu output. Should that fail, it attempts _cpu_name_from_devicetree to read the hardware identifier from the system device-tree.

macOS and Windows Fallbacks

• macOS: The tool executes sysctl -n machdep.cpu.brand_string to retrieve the CPU identifier directly from the kernel.
• Windows: The system attempts wmic cpu get name first, then falls back to PowerShell CIM queries via _cpu_name_from_windows_cim if the WMI call fails or returns incomplete data.

Physical Core Count Detection

The tool determines available compute resources using detect_cpu_cores() in src/whichllm/hardware/cpu.py (lines 154-168).

Primary method: Uses psutil.cpu_count(logical=False) to get the physical core count.

Fallback mechanism: If psutil returns None—observed in virtualized environments like WSL2—WhichLLM parses /proc/cpuinfo directly, counting unique combinations of physical id and core id pairs through the _count_physical_cores_linux helper function.

AVX Instruction Set Support

Vector instruction support is critical for CPU-based inference performance. WhichLLM detects AVX2 and AVX-512 capabilities via detect_avx_support() in src/whichllm/hardware/cpu.py (lines 176-226).

Linux and macOS Detection

• Linux: Scans the "flags" line in /proc/cpuinfo for avx2 and avx512f tokens.
• macOS: Queries sysctl for hw.optional.avx2_0 and hw.optional.avx512f boolean values to determine hardware support.

Windows Assumptions

On Windows, the code assumes AVX2 is present and AVX-512 is absent, serving as a conservative fallback when direct CPU feature detection is unavailable through standard system APIs.

Practical Implementation Examples

You can leverage WhichLLM's detection logic in your own scripts to gather hardware information:

from whichllm.hardware import cpu

# Get comprehensive CPU information

cpu_name = cpu.detect_cpu_name()
cpu_cores = cpu.detect_cpu_cores()
avx2, avx512 = cpu.detect_avx_support()

print(f"CPU: {cpu_name}")
print(f"Cores: {cpu_cores}")
print(f"AVX2 support: {avx2}")
print(f"AVX-512 support: {avx512}")

The hardware descriptor is consumed by src/whichllm/output/display.py to format CLI output and by src/whichllm/engine/compatibility.py to determine if a model can run in CPU-only mode without GPU acceleration.

Summary

• WhichLLM gathers three critical CPU data points: model name, physical core count, and AVX instruction set support.
• Detection logic resides in src/whichllm/hardware/cpu.py, with platform-specific implementations for Linux, macOS, and Windows.
• The CPU name detection uses a fallback chain: /proc/cpuinfo → lscpu → device-tree on Linux; sysctl on macOS; WMI → PowerShell CIM on Windows.
• Core counting prefers psutil but falls back to manual /proc/cpuinfo parsing on WSL2.
• AVX detection uses /proc/cpuinfo flags on Linux and sysctl hardware options on macOS, with conservative defaults on Windows.
• This data feeds into HardwareInfo objects used by the compatibility engine and CLI display formatter.

Frequently Asked Questions

How does WhichLLM detect CPU model names on ARM systems?

On ARM or aarch64 Linux systems where /proc/cpuinfo lacks a "model name" field, WhichLLM falls back to the _cpu_name_from_lscpu helper to extract the name from lscpu output. If that fails, it attempts _cpu_name_from_devicetree to read the hardware identifier from the system device-tree.

What happens if psutil cannot determine the physical core count?

When psutil.cpu_count(logical=False) returns None—which occurs in certain virtualized environments like WSL2—the detect_cpu_cores() function calls _count_physical_cores_linux. This helper parses /proc/cpuinfo to identify unique physical CPU and core ID combinations, accurately counting physical cores even without psutil support.

Why does WhichLLM check for AVX2 and AVX-512 support?

AVX2 and AVX-512 are SIMD (Single Instruction, Multiple Data) instruction sets that dramatically accelerate matrix operations used in LLM inference. By detecting these capabilities in detect_avx_support(), WhichLLM can filter model recommendations to only suggest variants that will actually run efficiently on the user's hardware, or warn when CPU-only inference will be prohibitively slow.

Can I use WhichLLM's CPU detection functions in my own Python scripts?

Yes. The whichllm.hardware.cpu module exposes public functions including detect_cpu_name(), detect_cpu_cores(), and detect_avx_support() that you can import directly. These functions return native Python types (strings, integers, booleans) and handle platform detection internally, making them reusable for any project requiring cross-platform CPU identification.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}