Debug kt-kernel Installation Issues Related to CPU Variant Detection

kt-kernel selects instruction-set variants (AMX, AVX-512, AVX2) at build time using a dual-detection system, and mismatches between detected capabilities and compiled binaries cause link errors, illegal instruction crashes, or silent performance degradation.

The kt-kernel component of the kvcache-ai/ktransformers repository compiles platform-specific optimized kernels for x86_64 CPUs. Because these kernels rely on advanced instruction sets like AMX (Advanced Matrix Extensions) and AVX-512, the build system must accurately detect CPU capabilities. When detection fails or the build environment differs from the runtime environment, installation breaks with cryptic errors. This guide explains how to diagnose and fix these CPU variant mismatches using the actual detection mechanisms in the source code.

How CPU Variant Detection Works

kt-kernel uses two complementary detection mechanisms that operate at different stages of the build process.

Runtime Feature Probe

The Python helper script kt-kernel/scripts/check_cpu_features.py analyzes /proc/cpuinfo at runtime to determine which instruction-set flags the CPU actually supports. According to lines 100-106 of this script, it prints a clear recommendation for which CMake flags you should use. This is your primary diagnostic tool when debugging installation failures.

CMake Compile-Time Detection

During the configuration phase, kt-kernel/cmake/DetectCPU.cmake probes the compiler and system to set cache variables. Lines 130-138 contain the critical logic:

if(NOT DEFINED KTRANSFORMERS_CPU_USE_AMX AND HAS_AMX)
    set(KTRANSFORMERS_CPU_USE_AMX ON CACHE BOOL "Enable AMX" FORCE)
endif()

Similar blocks handle AVX-512 detection. If these checks fail, the variables remain OFF and the build silently falls back to AVX2.

Recognizing CPU Variant Mismatch Symptoms

When the compiled binary's instruction set does not match the host CPU capabilities, you will encounter one of these specific failure modes:

• Link-time errors such as undefined reference to '__builtin_ia32_amx_*' — the binary was compiled expecting AMX instructions that the linker cannot resolve for the target CPU.
• Runtime aborts reporting illegal instruction (core dumped) — the binary contains AVX-512F, BF16, VNNI, or VBMI instructions that the CPU does not recognize.
• Silent performance degradation — the library loads successfully but executes AVX2 code paths despite the CPU supporting AVX-512, resulting in significantly slower inference.

Step-by-Step Debugging Workflow

Follow this exact sequence to identify and resolve the root cause of variant-related installation failures.

1. Run the Runtime CPU Probe

Execute the Python detection script to establish baseline capabilities:

python3 kt-kernel/scripts/check_cpu_features.py

The output reports:

• CPU Model (e.g., "Intel Xeon Gold 6348")
• AMX Support status for amx_tile, amx_int8, and amx_bf16
• AVX-512 Support for avx512f, avx512_bf16, avx512_vnni, and avx512_vbmi
• Recommendation block specifying exact CMake flags needed

2. Inspect CMake Detection Logic

Open kt-kernel/cmake/DetectCPU.cmake and verify that the detection logic around lines 130-138 correctly identified your CPU features. If the script failed to detect AMX or AVX-512 despite the CPU probe showing support, your Linux kernel may be hiding these flags (common on kernels older than 5.10).

3. Force the Correct Variant

If the CPU probe confirms support but CMake missed it, manually override the cache variables:

cmake -S . -B build \
    -DKTRANSFORMERS_CPU_USE_AMX=ON \
    -DKTRANSFORMERS_CPU_USE_AMX_AVX512=ON \
    -DKTRANSFORMERS_CPU_ARCH=x86_64

If the probe shows missing extensions, you must either use a pre-built AVX2 wheel, upgrade your hardware/firmware, or remove explicit -mavx512* flags from any custom build scripts.

4. Build and Verify Symbols

Compile the library and inspect the generated symbols to confirm the correct variant was baked in:

cmake --build build -j$(nproc)
nm -D build/libktkernel.so | grep vpdpbusds

The presence of vpdpbusds (VNNI) or AMX-related symbols confirms the high-performance path was compiled.

5. Validate the Installation

Re-run the CPU probe after installation to ensure the "Recommendation" section now shows a checkmark path matching your compiled variant.

Common Pitfalls and Solutions

Symptom	Root Cause	Solution
"Illegal instruction" on startup	Binary built for AMX/AVX-512 but CPU lacks flags	Run check_cpu_features.py → confirm missing flags → rebuild without -DKTRANSFORMERS_CPU_USE_AMX* or -DKTRANSFORMERS_CPU_USE_AMX_AVX512
CMake reports "AMX not detected" on known-AMX CPU	Linux kernel < 5.10 hides AMX in /proc/cpuinfo	Upgrade kernel or manually set -DKTRANSFORMERS_CPU_USE_AMX=ON
No AVX-512 symbols in compiled library	Stale CMake cache from previous configuration	Delete CMakeCache.txt and build/ directory, then reconfigure
AVX2-level performance on AVX-512 CPU	Missing avx512_bf16 or avx512_vbmi caused fallback	Verify flags via probe; if present, force -DKTRANSFORMERS_CPU_USE_AMX_AVX512=ON

Key Source Files

Understanding these specific files helps you trace detection logic:

• kt-kernel/scripts/check_cpu_features.py — Runtime detector that parses /proc/cpuinfo and recommends CMake flags.
• kt-kernel/cmake/DetectCPU.cmake — CMake module that sets KTRANSFORMERS_CPU_USE_AMX and related cache variables during configuration.
• kt-kernel/cmake/KTransformersConfig.cmake — Generated configuration exposing CPU variant options to downstream builds.
• setup.py — Python entry point that forwards CMake flags when building from source.
• kt-kernel/ext_bindings.cpp — Implementation of the CPUInfer class compiled with the selected CPU variant.

Summary

• Always run kt-kernel/scripts/check_cpu_features.py before building to establish your CPU's actual capabilities.
• CMake detection in DetectCPU.cmake sets KTRANSFORMERS_CPU_USE_AMX and KTRANSFORMERS_CPU_USE_AMX_AVX512; manually override these when the kernel hides CPU flags.
• Verify the build using nm -D to inspect for AVX-512 VNNI or AMX symbols, ensuring the optimized kernels were actually compiled.
• Delete stale caches (CMakeCache.txt) when switching between CPU variants to prevent silent fallbacks to AVX2.

Frequently Asked Questions

How do I know if my CPU supports AMX for kt-kernel?

Run the provided probe script: python3 kt-kernel/scripts/check_cpu_features.py. The script checks for amx_tile, amx_int8, and amx_bf16 flags in /proc/cpuinfo. If you see checkmarks for these flags, your CPU supports AMX. Note that older Linux kernels (below 5.10) may hide these flags even if the hardware supports them.

Why does kt-kernel crash with "illegal instruction" after installation?

This error occurs when the binary was compiled for a CPU variant (such as AVX-512 or AMX) that your current hardware does not support. Run the CPU feature probe to identify the mismatch, then rebuild from source without the unsupported flags (e.g., remove -DKTRANSFORMERS_CPU_USE_AMX=ON) or install the AVX2-only wheel instead.

Can I force kt-kernel to build for AVX-512 even if CMake doesn't detect it?

Yes. If you know your CPU supports AVX-512 but CMake fails to detect it (often due to /proc/cpuinfo formatting issues), manually set the cache variables: cmake -DKTRANSFORMERS_CPU_USE_AMX_AVX512=ON -DKTRANSFORMERS_CPU_USE_AMX=ON ... Always verify the resulting binary contains AVX-512 symbols using nm -D build/libktkernel.so | grep avx512 to confirm the override worked.

What should I do if CMake detects AMX but the build still fails?

If CMake sets KTRANSFORMERS_CPU_USE_AMX=ON but compilation fails with undefined references to __builtin_ia32_amx_*, your compiler toolchain likely lacks AMX intrinsics support. Ensure you are using GCC 11+ or Clang 14+. Alternatively, disable AMX by setting -DKTRANSFORMERS_CPU_USE_AMX=OFF and rely on AVX-512 or AVX2 instead.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}