Francois Chollet - Why The Biggest AI Models Can't Solve Simple Puzzles

Here is my conversation with Francois Chollet and Mike Knoop on the $1 million ARC-AGI Prize they're launching today.

I did a bunch of socratic grilling throughout, but Francois’s arguments about why LLMs won’t lead to AGI are very interesting and worth thinking through.

It was really fun discussing/debating the cruxes. Enjoy!

Check out ARC-AGI Prize here: https://arcprize.org

Links:
Transcript: https://www.dwarkeshpatel.com/p/francois-chollet
Apple Podcasts: https://podcasts.apple.com/us/podcast/francois-chollet-mike-knoop-llms-wont-lead-to-agi-%241/id1516093381?i=1000658672649
Spotify: https://open.spotify.com/episode/7bmeJQOvXGy4LYl6YoiYYP?si=obUSUEwjSA6tkB8EBcb18w

Follow me on Twitter: https://x.com/dwarkesh_sp

Timestamps:
00:00:00 – The ARC benchmark
00:11:53 – Why LLMs struggle with ARC
00:19:43 – Skill vs intelligence
00:28:38 – Do we need “AGI” to automate most jobs?
00:49:11 – Future of AI progress: deep learning + program synthesis
01:01:23 – How Mike Knoop got nerd-sniped by ARC
01:09:20 – Million $ ARC Prize
01:11:16 – Resisting benchmark saturation
01:18:51 – ARC scores on frontier vs open source models
01:27:02 – Possible solutions to ARC Prize