Learn how tokenization works in large language and embedding models and how the tokenizer can affect the quality of your search.
Beginner1h43m
Retrieval Optimization: From Tokenization to Vector Quantization
Instructor: Kacper Łukawski
Earn an accomplishment with PRO · Updated Jun 1, 2026
- Beginner
- 1h43m
- 8 Video Lessons
- 6 Code Examples
- 1 Graded Assignment PRO
- Earn an accomplishment with PRO
- Updated Jun 1, 2026
- Instructor: Kacper Łukawski
Qdrant- Learn more aboutMembership PRO Plan
What you'll learn
Explore how different tokenization techniques including Byte-Pair Encoding, WordPiece, and Unigram are trained and work.
Understand how to measure the quality of your retrieval and how to optimize your search by adjusting HNSW parameters and vector quantizations.
Course recap
PROThis course examines the full retrieval pipeline from tokenization to vector quantization, focusing on how each component affects retrieval quality and efficiency. It covers tokenizer training algorithms (BPE, WordPiece, Unigram, SentencePiece), their impact on embedding models, and vector compression techniques for scaling retrieval systems.
Concept map
Key concepts
- Tokenization algorithmsBPE (bottom-up character merging), WordPiece (prefix/suffix distinction with ## markers), Unigram (top-down vocabulary pruning), SentencePiece (whitespace-agnostic)
- EmbeddingsDense vector representations whose quality depends heavily on the underlying tokenizer
- Vector quantizationCompressing embedding vectors for storage and search efficiency
- QuantizationApplied to vectors rather than model weights; reduces memory footprint of embedding indices
- Vocabulary sizeHyperparameter (typically 30K+) that affects tokenization granularity; larger for multilingual models
- Glitch tokensTokens in BPE/WordPiece vocabularies that appear but are never used in actual tokenization; Unigram avoids this
Lesson highlights
- 1.Introduction — Overview of the retrieval pipeline from tokenization to search
- 2.Role of Tokenizers — Training BPE, WordPiece, and Unigram tokenizers from scratch with HuggingFace Tokenizers
- 3.Tokenizer Impact on Embeddings — How tokenizer choice affects embedding model performance
- 4.Vector Quantization Fundamentals — Scalar and product quantization for embedding compression
- 5.Advanced Quantization Methods — Multi-step quantization, residual quantization
- 6.Evaluation — Measuring retrieval quality with compressed vectors
- 7.Production Considerations — Scaling retrieval with quantized indices
- 8.Conclusion
About this course
In Retrieval Optimization: From Tokenization to Vector Quantization, taught by Kacper Łukawski, Developer Relations Lead of Qdrant, you’ll learn all about tokenization and also how to optimize vector search in your large-scale customer-facing RAG applications. You’ll explore the technical details of how vector search works and how to optimize it for better performance.
This course focuses on optimizing the first step in your RAG and search results. You’ll see how different tokenization techniques like Byte-Pair Encoding, WordPiece, and Unigram work and how they affect search relevancy. You’ll also learn how to address common challenges such as terminology mismatches and truncated chunks in embedding models.
To optimize your search, you need to be able to measure its quality. You will learn several quality metrics for this purpose. Most vector databases use Hierarchical Navigable Small Worlds (HNSW) for approximate nearest-neighbor search. You’ll see how to balance the HNSW parameters for higher speed and maximum relevance. Finally, you would use different vector quantization techniques to enhance memory usage and search speed.
What you’ll do, in detail:
- Learn about the internal workings of the embedding model and how your text is turned into vectors.
- Understand how several tokenizers such as Byte-Pair Encoding, WordPiece, Unigram, and SentencePiece are trained.
- Explore common challenges with tokenizers such as unknown tokens, domain-specific identifiers, and numerical values, that can negatively affect your vector search.
- Understand how to measure the quality of your search across several quality metrics
- Understand how the main parameters in HNSW algorithms affect the relevance and speed of vector search and how to optimally adjust these parameters.
- Experiment with the three major quantization methods, product, scalar, and binary, and learn how they impact memory requirements, search quality, and speed.
By the end of this course, you’ll have a solid understanding of how tokenization is done and how to optimize vector search in your RAG systems.
Who should join?
Anyone with basic Python knowledge who wants to learn to build effective customer-facing RAG applications!
Course Outline
8 Lessons・6 Code Examples- IntroductionVideo・6m
- Embedding modelsVideo with Code Example・16m
- Role of the tokenizersVideo with Code Example・15m
- Practical implications of the tokenizationVideo with Code Example・14m
- Measuring Search RelevanceVideo with Code Example・14m
- Optimizing HNSW searchVideo with Code Example・10m
- Vector quantizationVideo with Code Example・16m
- ConclusionVideo・1m
- Quiz
Graded・Quiz
・10m
Elevate your learning experience with Pro
Upgrade to Pro and gain unlimited accomplishments on your resume
Instructor
Course access is free for a limited time during the DeepLearning.AI learning platform beta!
Want to learn more about Generative AI?
Keep learning with updates on curated AI news, courses, and events, as well as Andrew’s thoughts from DeepLearning.AI!