Today, we're joined by Drago Anguelov, head of AI foundations at Waymo, for a deep dive into the role of foundation models in autonomous driving. Drago shares how Waymo is leveraging large-scale machine learning, including vision-language models and generative AI techniques to improve perception, planning, and simulation for its self-driving vehicles. The conversation explores the evolution of Waymo’s research stack, their custom “Waymo Foundation Model,” and how they’re incorporating multimodal sensor data like lidar, radar, and camera into advanced AI systems. Drago also discusses how Waymo ensures safety at scale with rigorous validation frameworks, predictive world models, and realistic simulation environments. Finally, we touch on the challenges of generalization across cities, freeway driving, end-to-end learning vs. modular architectures, and the future of AV testing through ML-powered simulation.
🎧 / 🎥 Listen or watch the full episode on our page: https://twimlai.com/go/725.
🔔 Subscribe to our channel for more great content just like this: https://youtube.com/twimlai?sub_confirmation=1
🗣️ CONNECT WITH US!
===============================
Subscribe to the TWIML AI Podcast: https://twimlai.com/podcast/twimlai/
Follow us on Twitter: https://twitter.com/twimlai
Follow us on LinkedIn: https://www.linkedin.com/company/twimlai/
Join our Slack Community: https://twimlai.com/community/
Subscribe to our newsletter: https://twimlai.com/newsletter/
Want to get in touch? Send us a message: https://twimlai.com/contact/
📖 CHAPTERS
===============================
00:00 - Introduction
3:56 - Waymo One
7:03 - Safety Impact
12:10 - VLMs and generative AI technologies
16:15 - EMMA
19:17 - VLMs
24:00 - Challenges
29:37 - Foundation model
30:52 - Sensor data integration and foundation models
36:18 - Relationship of sensor prediction and vehicle behavior
42:28 - End-to-end training vs. modular architectures
45:58 - Simulation, testing, and validation
49:22 - Explainability and controllability
52:25 - Testing process
55:20 - Challenges in defining rules for autonomous vehicles
58:57 - Simulation technologies
1:04:10 - Waymo Open Dataset Challenges
🔗 LINKS & RESOURCES
===============================
Waymo - https://waymo.com/
Waymo Safety Impact - https://waymo.com/safety/impact/
NVIDIA GTC Keynote: Advancing AI to Build the Most-Trusted Driver - https://register.nvidia.com/flow/nvidia/gtcs25/vap/page/vsessioncatalog/session/1727887107684001i7zG
System Design for Autonomous Vehicles with Drago Angelov - 454 - https://twimlai.com/podcast/twimlai/system-design-autonomous-vehicles-drago-angelov/
📸 Camera: https://amzn.to/3TQ3zsg
🎙️Microphone: https://amzn.to/3t5zXeV
🚦Lights: https://amzn.to/3TQlX49
🎛️ Audio Interface: https://amzn.to/3TVFAIq
🎚️ Stream Deck: https://amzn.to/3zzm7F5