# INT8, GPTQ, AWQ, and GGUF Quantization Methods in Phase 10 of AI Engineering From Scratch

> Explore Phase 10 of AI Engineering From Scratch to master INT8, GPTQ, AWQ, and GGUF quantization. Learn to deploy LLMs with 2-4x memory reduction for efficient AI.

- Repository: [Rohit Ghumare/ai-engineering-from-scratch](https://github.com/rohitg00/ai-engineering-from-scratch)
- Tags: deep-dive
- Published: 2026-08-29

---

**Phase 10 of the ai-engineering-from-scratch curriculum details four production quantization techniques—INT8, GPTQ, AWQ, and GGUF—that enable deploying large language models with 2× to 4× memory reduction through integer scaling, Hessian-guided layer optimization, activation-aware weight protection, and CPU-optimized mixed-precision formats.**

The "Quantization" lesson (Lesson 11) in **Phase 10** of the `rohitg00/ai-engineering-from-scratch` repository provides concrete implementations of model compression strategies widely used in inference engines like vLLM, TensorRT-LLM, and llama.cpp. According to the curriculum documentation in [`phases/10-llms-from-scratch/11-quantization/docs/en.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/phases/10-llms-from-scratch/11-quantization/docs/en.md), these methods balance memory efficiency against accuracy loss through distinct mathematical approaches ranging from symmetric integer quantization to salient-weight preservation algorithms.

## INT8 Quantization: Uniform 8-Bit Compression

**INT8 quantization** converts 32-bit floating-point weights into 8-bit integers using a symmetric scale factor, achieving approximately **2× memory reduction** with less than 0.5% quality degradation. As documented in `phases/10-llms-from-scratch/11-quantization/docs/en.md#L53-L55`, this method utilizes 256 uniform steps (values -127 to 127) with a per-tensor scaling factor derived from the maximum absolute value.

This approach is optimized for **GPU inference** via frameworks like vLLM and TensorRT-LLM, where the reduced memory bandwidth directly translates to higher throughput. The quantization process involves calculating a scale factor, clipping values to the INT8 range, and converting back to floating-point during dequantization.

### NumPy Implementation

The lesson provides a pure NumPy implementation demonstrating symmetric quantization:

```python
import numpy as np

def quantize_int8(tensor):
    qmin, qmax = -127, 127
    scale = np.max(np.abs(tensor)) / qmax
    quantized = np.clip(np.round(tensor / scale), qmin, qmax).astype(np.int8)
    return quantized, scale

def dequantize_int8(q, scale):
    return q.astype(np.float32) * scale

```

## GPTQ: Hessian-Guided INT4 Quantization

**GPTQ (General-purpose Post-Training Quantization)** enables aggressive **4-bit (INT4)** compression by using a small calibration set to compute Hessian-guided per-layer scaling factors. As detailed in `phases/10-llms-from-scratch/11-quantization/docs/en.md#L59-L61`, this post-training quantization (PTQ) technique is essential when deploying 70B parameter models on single 48GB GPUs, achieving roughly 4× memory savings compared to FP16.

GPTQ analyzes the layer-wise Hessian matrix to determine optimal scaling that minimizes the error introduced by quantization, processing weights in arbitrary order while accounting for quantization errors propagated to subsequent weights.

### AutoGPTQ Implementation

The curriculum demonstrates GPTQ using the `auto-gptq` library with a 4-bit configuration and group size of 128:

```python

# pip install auto-gptq transformers

from auto_gptq import AutoGPTQForCausalLM, BaseQuantizeConfig
from transformers import AutoTokenizer

model_id = "meta-llama/Llama-3.1-8B"
quantize_cfg = BaseQuantizeConfig(bits=4, group_size=128, desc_act=False)

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoGPTQForCausalLM.from_pretrained(model_id, quantize_cfg)

# 128-example calibration set (omitted for brevity)

model.quantize(calibration)
model.save_quantized("llama-8b-gptq-int4")

```

## AWQ: Activation-Aware Weight Protection

**AWQ (Activation-Aware Weight Quantization)** improves upon GPTQ by identifying and protecting the approximately **1% most salient weights**—those that multiply large activations—scaling them up before quantization to preserve model quality. According to `phases/10-llms-from-scratch/11-quantization/docs/en.md#L61-L62`, this method achieves comparable INT4 compression to GPTQ while being **1.5–2× faster to apply** during the quantization process.

The activation-aware approach recognizes that not all weights contribute equally to model output; by preserving high-impact weights in higher precision while quantizing the remainder, AWQ maintains accuracy without requiring the computational overhead of Hessian matrix calculations.

### AutoAWQ Implementation

The lesson provides an `autoawq` implementation demonstrating zero-point quantization with 128-size groups:

```python

# pip install autoawq

from awq import AutoAWQForCausalLM
from transformers import AutoTokenizer

model_id = "meta-llama/Llama-3.1-8B"
model = AutoAWQForCausalLM.from_pretrained(model_id)
tokenizer = AutoTokenizer.from_pretrained(model_id)

model.quantize(tokenizer, quant_config={"zero_point": True,
                                        "q_group_size": 128,
                                        "w_bit": 4})
model.save_quantized("llama-8b-awq-int4")

```

## GGUF: Mixed-Precision CPU Inference

**GGUF (GPT-Generated Unified Format)** is the native format for [`llama.cpp`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/llama.cpp), packing mixed-precision weights (such as **Q4_K_M** and **Q5_K_M**) into a single file optimized for CPU or Apple Silicon inference. As described in `phases/10-llms-from-scratch/11-quantization/docs/en.md#L63-L65`, this format is specifically designed for **edge devices and laptops** where GPU acceleration is unavailable.

Unlike uniform quantization methods, GGUF employs different quantization strategies for different tensor types (attention queries vs. feed-forward networks) to optimize the quality-to-size ratio for CPU inference engines.

### llama.cpp Conversion and Serving

The curriculum demonstrates converting HuggingFace models to GGUF format and serving them:

```bash

# pip install llama-cpp-python

python convert_hf_to_gguf.py meta-llama/Llama-3.1-8B \
    --outtype q4_k_m \
    --outfile llama-8b-q4km.gguf

# Serve with the llama.cpp server

llama-server -m llama-8b-q4km.gguf -c 4096 -ngl 99

```

## Selecting the Right Quantization Method

The decision framework presented in [`phases/10-llms-from-scratch/11-quantization/outputs/skill-quantization.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/phases/10-llms-from-scratch/11-quantization/outputs/skill-quantization.md) recommends selecting quantization methods based on hardware constraints and quality requirements:

- **INT8**: Use for GPU deployments requiring maximum speed with minimal accuracy loss
- **GPTQ**: Use for aggressive GPU memory reduction when calibration data is available
- **AWQ**: Use for faster quantization workflows while maintaining GPTQ-level quality
- **GGUF**: Use for CPU-only inference environments and edge deployment

## Summary

- **INT8 quantization** provides a 2× memory reduction through symmetric 8-bit integer representation with <0.5% quality loss, ideal for high-throughput GPU inference
- **GPTQ** enables 4-bit compression via Hessian-guided optimization, making 70B models viable on consumer GPUs through post-training calibration
- **AWQ** accelerates the quantization process by protecting salient weights based on activation magnitudes, offering comparable compression to GPTQ with faster processing times
- **GGUF** delivers mixed-precision CPU-optimized formats through llama.cpp, supporting edge deployment on laptops and Apple Silicon devices
- All four methods are implemented with runnable code examples in [`phases/10-llms-from-scratch/11-quantization/code/quantization_demo.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/phases/10-llms-from-scratch/11-quantization/code/quantization_demo.py) and documented in the Phase 10 quantization lesson

## Frequently Asked Questions

### What is the difference between GPTQ and AWQ quantization?

**GPTQ** uses a Hessian matrix derived from calibration data to guide per-layer quantization scales, while **AWQ** identifies the top 1% of weights by activation magnitude and scales them up before quantization to preserve accuracy. According to the Phase 10 curriculum in [`phases/10-llms-from-scratch/11-quantization/docs/en.md`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/phases/10-llms-from-scratch/11-quantization/docs/en.md), AWQ achieves similar compression ratios to GPTQ but processes approximately 1.5–2× faster because it avoids computationally expensive Hessian calculations.

### When should I use GGUF instead of GPTQ or AWQ?

Use **GGUF** when deploying models on **CPU-only environments** or edge devices such as laptops and Apple Silicon Macs, as the format is specifically optimized for llama.cpp inference engines. **GPTQ** and **AWQ** are designed for GPU acceleration and will not run efficiently on CPU-only systems, whereas GGUF formats like Q4_K_M provide mixed-precision compression tailored for CPU memory bandwidth constraints.

### Does INT8 quantization require retraining the model?

No, **INT8 quantization** is a post-training technique that applies symmetric scaling to pre-trained weights without requiring gradient updates or fine-tuning. As implemented in [`phases/10-llms-from-scratch/11-quantization/code/quantization_demo.py`](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/phases/10-llms-from-scratch/11-quantization/code/quantization_demo.py), the method calculates a simple per-tensor scale factor based on maximum absolute values, enabling immediate deployment with less than 0.5% accuracy degradation.

### Which quantization method offers the best memory reduction?

**GPTQ** and **AWQ** both achieve **4-bit (INT4)** compression, providing approximately **4× memory reduction** compared to FP16 models, making them optimal for maximizing memory efficiency. While GGUF also supports 4-bit variants like Q4_K_M, the uniform INT8 format offers only 2× reduction but maintains higher precision for applications where accuracy is critical.