This course is no longer available as of June 26, 2025. Explore similar courses on Multimodal and RAG Applications.
Intermediate1h10m
Multimodal RAG: Chat with Videos
Instructor: Vasudev Lal
Earn an accomplishment with PRO · Updated Sep 3, 2025
- Intermediate
- 1h10m
- 8 Video Lessons
- 6 Code Examples
- 1 Graded Assignment PRO
- Earn an accomplishment with PRO
- Updated Sep 3, 2025
- Instructor: Vasudev Lal
Intel- Learn more aboutMembership PRO Plan
Course Outline
8 Lessons・6 Code Examples- IntroductionVideo・4m
- Interactive Demo and Multimodal RAG System ArchitectureVideo with Code Example・7m
- Multimodal EmbeddingsVideo with Code Example・9m
- Preprocessing Videos for Multimodal RAGVideo with Code Example・9m
- Multimodal Retrieval from Vector StoresVideo with Code Example・6m
- Large Vision - Language Models (LVLMs)Video with Code Example・7m
- Multimodal RAG with Multimodal LangchainVideo with Code Example・13m
- ConclusionVideo・1m
- Quiz
Graded・Quiz
・10m
Elevate your learning experience with Pro
Upgrade to Pro and gain unlimited accomplishments on your resume
Course recap
PROThis course, built with Intel, teaches how to build a question-answering system for video collections using multimodal RAG. It covers multimodal encoding with the BridgeTower model (which embeds both text and images into a unified semantic space), video processing with Whisper transcription and vision-language captioning, vector storage and retrieval, and end-to-end RAG pipelines feeding retrieved content to large vision-language models.
Concept map
Key concepts
- EmbeddingsJoint multimodal embeddings from the BridgeTower model that represent both vision and language in a common semantic space
- Retrieval-Augmented GenerationRetrieval-augmented generation applied to video data, fetching relevant frames and transcripts for LLM-grounded responses
- Multimodal ModelsModels that process and align image and text data, including BridgeTower for embedding and large vision-language models for generation
Lesson highlights
- 1.Introduction — Overview of multimodal RAG for video and the course architecture
- 2.Interactive Demo — Hands-on with the completed Gradio-based video chat system
- 3.BridgeTower Model — Creating joint embeddings for image-caption pairs in a unified semantic space
- 4.Video Processing — Transcription with Whisper, frame extraction, and captioning with vision-language models
- 5.Vector Storage — Storing multimodal embeddings in vector databases for retrieval
- 6.Retrieval Pipeline — Building retrieval systems for complex multimodal queries using LangChain
- 7.Generation and Chat — Using large vision-language models to generate grounded responses from retrieved context
- 8.Conclusion — Complete system architecture and future directions
Instructor
Course access is free for a limited time during the DeepLearning.AI learning platform beta!
Want to learn more about Generative AI?
Keep learning with updates on curated AI news, courses, and events, as well as Andrew’s thoughts from DeepLearning.AI!