Quantization (ML)
Model compression technique
, from 32-bit floating point to 8-bit integers), shrinking model size and speeding up inference with minimal accuracy loss.
Definition
Quantization is a model compression technique that reduces the numerical precision used to represent a neural network's weights and activations (e.g., from 32-bit floating point to 8-bit integers), shrinking model size and speeding up inference with minimal accuracy loss.
Overview
Neural networks are typically trained using 32-bit floating-point numbers (FP32) for weights and activations, which provides high numerical precision but consumes significant memory and computational resources. Quantization reduces this precision — commonly to 16-bit floating point (FP16/BF16), 8-bit integers (INT8), or even lower bit-widths like 4-bit — mapping the original range of floating-point values onto a smaller discrete set of representable values, typically via a scale factor and, for asymmetric schemes, a zero-point offset. This can reduce model size by 4x (FP32 to INT8) or more, and significantly speed up inference on hardware with optimized low-precision arithmetic units, such as modern GPU tensor cores and mobile NPUs. Two main approaches exist: post-training quantization (PTQ), which quantizes an already-trained model's weights (and calibrates activation ranges using a small representative dataset) without further training, offering a fast and simple compression path with some accuracy loss; and quantization-aware training (QAT), which simulates quantization effects during training itself (using "fake quantization" nodes that round values but still allow gradients to flow), letting the model adapt its weights to be more robust to the eventual precision reduction, typically preserving accuracy better than PTQ at the cost of requiring additional training compute. Quantization has become especially critical for deploying large language models, where techniques like GPTQ, AWQ, and GGUF-format quantization allow multi-billion-parameter models to run on consumer GPUs or even CPUs by reducing memory footprint dramatically, often to 4 bits per weight or less with careful calibration to preserve most of the model's output quality. Beyond LLMs, quantization is essential for deploying any model on resource-constrained edge devices — smartphones, embedded systems, and IoT hardware — where memory, power, and compute budgets are tightly limited, and is commonly used alongside pruning and knowledge distillation as one of the core model-compression techniques in production machine learning.
Key Concepts
- Reduces numerical precision of weights/activations (e.g., FP32 to INT8)
- Shrinks model size, often 4x or more, and speeds up inference
- Post-training quantization (PTQ) applies compression after training completes
- Quantization-aware training (QAT) simulates quantization during training for better accuracy
- Enables large language models to run on consumer GPUs or CPUs via 4-bit formats like GGUF
- Techniques like GPTQ and AWQ specialize in calibrated LLM weight quantization
- Critical for deploying models on resource-constrained edge and mobile devices
- Commonly combined with pruning and knowledge distillation for compression
Use Cases
Frequently Asked Questions
From the Blog
LLM Quantization Explained: Smaller, Faster Models
Quantization shrinks a language model by storing its numbers at lower precision, cutting memory and speeding it up with only a small loss in quality.
Read More AI & TechnologyWhat Is Model Quantization and Why It Matters
Model quantization shrinks a neural network by storing its weights in lower precision, cutting memory and speeding inference with little accuracy loss.
Read More ProgrammingQuantization Explained: Shrinking Models Without Losing Power
Quantization reduces the numerical precision of a model's weights so it runs faster and fits in less memory. This guide explains how the quantization parameter works, why it matters for deploying AI, and how to choose the right precision level.
Read More