# What Models Are Officially Supported by BitNet: Complete List and Setup Guide

> Discover the officially supported BitNet models including bitnet_b1_58-large, Llama3-8B, and Falcon3. Get a complete list and setup guide for x86 and ARM architectures.

- Repository: [Microsoft/BitNet](https://github.com/microsoft/BitNet)
- Tags: getting-started
- Published: 2026-03-13

---

**BitNet officially supports five model families—including bitnet_b1_58-large, bitnet_b1_58-3B, Llama3-8B-1.58-100B-tokens, Falcon3, and Falcon-E—each with specific kernel implementations for x86 and ARM architectures.**

The `microsoft/BitNet` repository provides a curated list of verified models that work with the **bitnet.cpp** inference engine. These models are pre-configured with optimized kernels and can be downloaded automatically via the provided setup scripts.

## Officially Supported Models

The repository maintains a definitive compatibility matrix in [`README.md`](https://github.com/microsoft/BitNet/blob/main/README.md) (lines 86-154) that enumerates which models work with the CPU and GPU kernels. According to the source code, the following models are officially supported:

| Model | Parameters | x86 Support | ARM Support |
|-------|------------|-------------|-------------|
| **bitnet_b1_58-large** | 0.7 B | I2_S, TL2 | TL1, TL2 |
| **bitnet_b1_58-3B** | 3.3 B | TL2 | TL1 |
| **Llama3-8B-1.58-100B-tokens** | 8.0 B | I2_S, TL2 | TL1, TL2 |
| **Falcon3 Family** | 1 B – 10 B | I2_S, TL2 | TL1, TL2 |
| **Falcon-E Family** | 1 B – 3 B | I2_S, TL2 | TL1, TL2 |

The checkmarks indicate which quantization kernels are implemented and tested for each hardware platform. **I2_S** represents 2-bit integer weights with 16-bit accumulators, while **TL1** and **TL2** denote Ternary-Lookup kernels optimized for different memory-throughput trade-offs.

## Kernel Support Matrix

Each supported model works with specific backend implementations in `src/ggml-bitnet-*.cpp` and `gpu/bitnet_kernels/bitnet_kernels.cu`. The kernel availability varies by architecture:

- **I2_S**: Available for bitnet_b1_58-large, Llama3-8B, Falcon3, and Falcon-E on x86; not available on ARM
- **TL1**: Available for bitnet_b1_58-3B and all models on ARM; not available on x86 for most models
- **TL2**: Universal support across all models on x86; available for bitnet_b1_58-large, Llama3-8B, Falcon3, and Falcon-E on ARM

The [`setup_env.py`](https://github.com/microsoft/BitNet/blob/main/setup_env.py) script (lines 14-18) validates these combinations when you specify a Hugging Face repository identifier.

## How to Install Officially Supported Models

The [`setup_env.py`](https://github.com/microsoft/BitNet/blob/main/setup_env.py) utility restricts downloads to verified Hugging Face repositories. Valid `--hf-repo` values include:

- `1bitLLM/bitnet_b1_58-large`
- `1bitLLM/bitnet_b1_58-3B`
- `HF1BitLLM/Llama3-8B-1.58-100B-tokens`
- `tiiuae/Falcon3-1B-Instruct-1.58bit` (and other Falcon3 variants)

To download an officially supported model:

```bash

# Install dependencies

conda create -n bitnet-cpp python=3.9
conda activate bitnet-cpp
pip install -r requirements.txt

# Download a verified model

python setup_env.py --hf-repo 1bitLLM/bitnet_b1_58-large

```

This command triggers automatic kernel generation via [`utils/codegen_tl1.py`](https://github.com/microsoft/BitNet/blob/main/utils/codegen_tl1.py) and [`utils/codegen_tl2.py`](https://github.com/microsoft/BitNet/blob/main/utils/codegen_tl2.py), creating optimized lookup-table headers in `preset_kernels/bitnet_b1_58-large/`.

## Running Inference on Supported Models

After installation, you can run inference using the pre-generated kernels. Each model ships with preset kernel configurations in `preset_kernels/` directories containing tuned `kernel_config_*.ini` files.

### CPU Inference

Use [`run_inference.py`](https://github.com/microsoft/BitNet/blob/main/run_inference.py) (defaulting to `models/bitnet_b1_58-3B/ggml-model-i2_s.gguf` per lines 45-48):

```bash
python run_inference.py \
    -m models/bitnet_b1_58-large/ggml-model-i2_s.gguf \
    -p "Explain the benefits of 1-bit LLMs."

```

To use TL1 or TL2 kernels instead, reference the corresponding GGUF file generated by the code generation utilities.

### GPU Inference

For CUDA-accelerated inference, use [`run_inference_server.py`](https://github.com/microsoft/BitNet/blob/main/run_inference_server.py) (lines 49-55):

```bash
python run_inference_server.py \
    -m models/bitnet_b1_58-large/ggml-model-tl2.gguf \
    --port 8080

```

Query the server via HTTP:

```bash
curl -X POST http://localhost:8080/generate \
     -d '{"prompt":"Summarize the BitNet paper in 2 sentences."}'

```

## Model Architecture and Kernel Generation

Each officially supported model includes architecture-specific parameters that drive kernel generation. The [`utils/codegen_tl1.py`](https://github.com/microsoft/BitNet/blob/main/utils/codegen_tl1.py) and [`utils/codegen_tl2.py`](https://github.com/microsoft/BitNet/blob/main/utils/codegen_tl2.py) scripts automatically produce lookup-table headers (e.g., [`bitnet-lut-kernels-tl1.h`](https://github.com/microsoft/BitNet/blob/main/bitnet-lut-kernels-tl1.h)) based on the model's hidden size and sequence length.

For testing without downloading full checkpoints, use the dummy model generator:

```bash
python utils/generate-dummy-bitnet-model.py \
    models/bitnet_b1_58-large \
    --outfile models/dummy-125m.tl1.gguf \
    --outtype tl1 \
    --model-size 125M

```

This creates a minimal GGUF file compatible with [`run_inference.py`](https://github.com/microsoft/BitNet/blob/main/run_inference.py) for pipeline verification.

## Summary

- **Five model families** are officially supported: bitnet_b1_58-large (0.7B), bitnet_b1_58-3B (3.3B), Llama3-8B-1.58-100B-tokens (8B), Falcon3 (1B-10B), and Falcon-E (1B-3B).
- **Kernel availability** varies by architecture: x86 supports I2_S and TL2 for most models, while ARM primarily supports TL1 and TL2.
- **Installation** requires using [`setup_env.py`](https://github.com/microsoft/BitNet/blob/main/setup_env.py) with specific Hugging Face repository identifiers defined in the source code.
- **Inference scripts** [`run_inference.py`](https://github.com/microsoft/BitNet/blob/main/run_inference.py) (CPU) and [`run_inference_server.py`](https://github.com/microsoft/BitNet/blob/main/run_inference_server.py) (GPU) automatically select appropriate kernels from `preset_kernels/` directories.

## Frequently Asked Questions

### Can I use custom models with BitNet?

No, the [`setup_env.py`](https://github.com/microsoft/BitNet/blob/main/setup_env.py) script strictly validates `--hf-repo` arguments against the hardcoded dictionary in lines 14-18. While you can manually convert models using the quantization utilities, only the five officially supported families are guaranteed to work with the pre-generated kernels in `preset_kernels/`.

### Which kernel should I use for maximum performance on x86?

**TL2** (Ternary-Lookup-2) generally provides the best throughput on modern x86 vector units. According to the README compatibility table, TL2 is available for all supported models on x86 architectures, while I2_S is limited to specific models.

### How do I verify my model is running the correct kernel?

Check the generated header files in `preset_kernels/[model_name]/` after running [`setup_env.py`](https://github.com/microsoft/BitNet/blob/main/setup_env.py). The presence of [`bitnet-lut-kernels-tl1.h`](https://github.com/microsoft/BitNet/blob/main/bitnet-lut-kernels-tl1.h) or [`bitnet-lut-kernels-tl2.h`](https://github.com/microsoft/BitNet/blob/main/bitnet-lut-kernels-tl2.h) confirms which lookup-table kernels were generated for your specific model architecture.

### Are quantized Falcon models compatible with both CPU and GPU?

Yes, the Falcon3 and Falcon-E families support both CPU kernels (I2_S, TL2 on x86; TL1, TL2 on ARM) and CUDA kernels via [`run_inference_server.py`](https://github.com/microsoft/BitNet/blob/main/run_inference_server.py). The same GGUF files work across both backends, though performance characteristics differ based on the selected kernel type.