Learn to deploy AI models on edge devices like smartphones, using their local compute power for faster and more secure inference.
Beginner1h10m
Introduction to on-device AI
Instructor: Krishna Sridhar
Earn an accomplishment with PRO · Updated Oct 7, 2025
- Beginner
- 1h10m
- 7 Video Lessons
- 4 Code Examples
- 1 Graded Assignment PRO
- Earn an accomplishment with PRO
- Updated Oct 7, 2025
- Instructor: Krishna Sridhar
Qualcomm- Learn more aboutMembership PRO Plan
What you'll learn
Explore model conversion by, converting your PyTorch/TensorFlow models for device compatibility, and quantize them to achieve performance gains while reducing model size.
Learn about device integration, including runtime dependencies, and how GPU, NPU, and CPU compute unit utilization affect performance.
Course recap
PROThis course teaches how to deploy AI models on edge devices such as smartphones, covering the full pipeline from cloud-trained models to real-time on-device inference. Taught by Krishna Sridhar from Qualcomm, it focuses on practical deployment using the Qualcomm AI Hub platform with real physical devices provisioned in the cloud.
Concept map
Key concepts
- QuantizationReducing model precision from float32 to int8/int4, achieving up to 4x smaller models and 4x faster inference through post-training quantization or quantization-aware training
- On device inferenceRunning AI models directly on edge devices using CPUs, GPUs, or NPUs for low latency, privacy, and cost efficiency
- Neural processing unitSpecialized hardware (NPU) on modern devices that is up to 10x more efficient than CPU for neural network workloads
- Model compilationConverting PyTorch models to device-compatible formats (TFLite, ONNX) through graph capture and target-specific compilation
Lesson highlights
- 1.Introduction — Overview of on-device AI deployment and its growing importance across billions of devices
- 2.Why On-Device — Benefits (cost, efficiency, privacy, personalization), real-world applications, and the four-step deployment process
- 3.Deploying Segmentation Models On-Device — First hands-on deployment of FFNET model with performance profiling on Samsung Galaxy S23
- 4.Preparing for On-Device Deployment — Deep dive into graph capture, compilation, hardware acceleration, and numerical validation
- 5.Quantizing Models — Post-training quantization with calibration data, achieving 3.7x speedup and 4x size reduction
- 6.Device Integration — Integrating models into camera pipelines with GPU pre/post-processing and NPU inference
- 7.Conclusion — Summary and next steps
About this course
As AI moves beyond the cloud, on-device inference is rapidly expanding to smartphones, IoT devices, robots, AR/VR headsets, and more. Billions of mobile and other edge devices are ready to run optimized AI models.
This course equips you with key skills to deploy AI on device:
- Explore how deploying models on device reduces latency, enhances efficiency, and preserves privacy.
- Go through key concepts of on-device deployment such as neural network graph capture, on-device compilation, and hardware acceleration.
- Convert pretrained models from PyTorch and TensorFlow for on-device compatibility.
- Deploy a real-time image segmentation model on device with just a few lines of code.
- Test your model performance and validate numerical accuracy when deploying to on-device environments
- Quantize and make your model up to 4x faster and 4x smaller for higher on-device performance.
- See a demonstration of the steps for integrating the model into a functioning Android app.
Learn from Krishna Sridhar, Senior Director of Engineering at Qualcomm, who has played a pivotal role in deploying over 1,000 models on devices and, with his team, has created the infrastructure used by over 100,000 applications.
By learning these techniques, you’ll be positioned to develop and deploy AI to billions of devices and optimize your complex models to run efficiently on the edge.
Who should join?
This course is designed for beginner AI developers, ML engineers, data scientists, and mobile developers looking to deploy optimized models on edge devices. Familiarity with Python, as well as PyTorch or TensorFlow is recommended.
Course Outline
7 Lessons・4 Code Examples- IntroductionVideo・4m
- Why on-deviceVideo・5m
- Deploying Segmentation Models On-DeviceVideo with Code Example・15m
- Preparing for on-device deploymentVideo with Code Example・14m
- Quantizing ModelsVideo with Code Example・13m
- Device IntegrationVideo・6m
- ConclusionVideo・1m
- Quiz
Graded・Quiz
・10m - Appendix - Building the AppCode Example・1m
Elevate your learning experience with Pro
Upgrade to Pro and gain unlimited accomplishments on your resume
Instructor
Course access is free for a limited time during the DeepLearning.AI learning platform beta!
Want to learn more about Generative AI?
Keep learning with updates on curated AI news, courses, and events, as well as Andrew’s thoughts from DeepLearning.AI!