Learn how to compress models with the Hugging Face Transformers library and the Quanto library.
Beginner1h14m
Quantization Fundamentals
Instructors: Younes Belkada, Marc Sun
Earn an accomplishment with PRO · Updated Oct 7, 2025
- Beginner
- 1h14m
- 7 Video Lessons
- 3 Code Examples
- 1 Graded Assignment PRO
- Earn an accomplishment with PRO
- Updated Oct 7, 2025
- Instructors: Younes Belkada, Marc Sun
Hugging Face- Learn more aboutMembership PRO Plan
What you'll learn
Learn about linear quantization, a simple yet effective method for compressing models.
Practice quantizing open source multimodal and language models.
Course recap
PROThis course introduces quantization as a technique for making large AI models more accessible by reducing their memory footprint. It covers data types in machine learning (FP32, FP16, BF16, INT8), the principles of linear quantization, hands-on quantization with HuggingFace's Quanto library, and recent advances in LLM quantization.
Concept map
Key concepts
- QuantizationRepresenting model weights in lower precision to reduce memory; FP32 (4 bytes) to INT8 (1 byte) achieves 4x compression
- Data typesFP32 (default), FP16 (half-precision, GPU-only on PyTorch), BF16 (CPU+GPU compatible, virtually no performance loss), INT8
- Linear quantizationMapping floating-point ranges to integer ranges using scale and zero-point parameters
- Model compressionThree main techniques: quantization, pruning (removing unimportant weights), and knowledge distillation (training smaller models from larger ones)
- Memory footprintmodel.get_memory_footprint() reveals actual memory usage; BF16 halves FP32 footprint
Lesson highlights
- 1.Handling Big Models — Why quantization matters; model size growth vs. hardware limits; compression techniques overview
- 2.Data Types — Integers (INT8), floating-point (FP32, FP16, BF16); precision vs. range tradeoffs
- 3.Loading Models by Data Type — Loading BLIP in FP32 vs. BF16; torch_dtype parameter; set_default_dtype for direct low-precision loading
- 4.Quantization Theory — Linear quantization formula; scale and zero-point computation
- 5.Quantizing with Quanto — HuggingFace Quanto library for INT8 quantization of transformer models
- 6.Recent Advances — State-of-the-art quantization methods for LLMs
- 7.Conclusion
About this course
Generative AI models, like large language models, often exceed the capabilities of consumer-grade hardware and are expensive to run. Compressing models through methods such as quantization makes them more efficient, faster, and accessible. This allows them to run on a wide variety of devices, including smartphones, personal computers, and edge devices, and minimizes performance degradation.
Join this course to:
- Quantize any open source model with linear quantization using the Quanto library.
- Get an overview of how linear quantization is implemented. This form of quantization can be applied to compress any model, including LLMs, vision models, etc.
- Apply “downcasting,” another form of quantization, with the Transformers library, which enables you to load models in about half their normal size in the BFloat16 data type.
By the end of this course, you will have a foundation in quantization techniques and be able to apply them to compress and optimize your own generative AI models, making them more accessible and efficient.
Who should join?
This is an introduction to the fundamental concepts of quantization for learners with a basic understanding of machine learning concepts and some experience with PyTorch, who is interested in learning about model quantization in generative AI.
Course Outline
7 Lessons・3 Code Examples
Elevate your learning experience with Pro
Upgrade to Pro and gain unlimited accomplishments on your resume
Instructors
Course access is free for a limited time during the DeepLearning.AI learning platform beta!
Want to learn more about Generative AI?
Keep learning with updates on curated AI news, courses, and events, as well as Andrew’s thoughts from DeepLearning.AI!