In this episode, we dive deep into the world of AI engineering with Chip Huyen, author of the excellent, newly released book "AI Engineering: Building Applications with Foundation Models".
We explore the nuances of AI engineering, distinguishing it from traditional machine learning, discuss how foundational models make it possible for anyone to build AI applications, and cover many other topics including the challenges of AI evaluation, the intricacies of the generative AI stack, why prompt engineering is underrated, why the rumors of the death of RAG are greatly exaggerated, and the latest progress in AI agents.
Book: https://www.oreilly.com/library/view/ai-engineering/9781098166298/
Chip Huyen
Website - https://huyenchip.com
LinkedIn - https://www.linkedin.com/in/chiphuyen
Twitter/X - https://x.com/chipro
FIRSTMARK
Website - https://firstmark.com
Twitter - https://twitter.com/FirstMarkCap
Matt Turck (Managing Director)
LinkedIn - https://www.linkedin.com/in/turck/
Twitter - https://twitter.com/mattturck
LISTEN ON:
Spotify - https://open.spotify.com/show/7yLATDSaFvgJG80ACcRJtq
Apple - https://podcasts.apple.com/us/podcast/the-mad-podcast-with-matt-turck/id1686238724
00:00 - Intro
02:45 - What is new about AI engineering?
06:11 - The product-first approach to building AI applications
07:38 - Are AI engineering and ML engineering two separate professions?
11:00 - The Generative AI stack
13:00 - Why are language models able to scale?
14:45 - Auto-regressive vs. masked models
16:46 - Supervised vs. unsupervised vs. self-supervised
18:56 - Why does model scale matter?
20:40 - Mixture of Experts
24:20 - Pre-training vs. post-training
28:43 - Sampling
32:14 - Evaluation as a key to AI adoption
36:03 - Entropy
40:05 - Evaluating AI systems
43:21 - AI as a judge
46:49 - Why prompt engineering is underrated
49:38 - In-context learning
51:46 - Few-shot learning and zero-shot learning
52:57 - Defensive prompt engineering
55:29 - User prompt vs. system prompt
57:07 - Why RAG is here to stay
01:00:31 - Defining AI agents
01:04:04 - AI agent planning
01:08:32 - Training data as a bottleneck to agent planning