On the first episode of the Pragmatic Engineer Podcast, I am joined by Simon Willison. Simon is one of the best-known software engineers experimenting with LLMs to boost his own productivity: he’s been doing this for more than three years, blogging about it in the open.
Simon is the creator of Datasette, an open-source tool for exploring and publishing data. He works full-time developing open-source tools for data journalism, centered on Datasette and SQLite. Previously, he was an engineering director at Eventbrite, joining through the acquisition of Lanyrd, a Y Combinator startup he co-founded in 2010. Simon is also a co-creator of the Django Web Framework. He has been blogging about web development since the early 2000s.
In today’s conversation, we dive deep into the realm of Gen AI and talk about the following:
• Simon’s initial experiments with LLMs and coding tools
• Why fine-tuning is generally a waste of time—and when it’s not
• RAG: an overview
• Interacting with GPTs voice mode
• Simon’s day-to-day LLM stack
• Common misconceptions about LLMs and ethical gray areas
• How Simon’s productivity has increased and his generally optimistic view on these tools
• Tips, tricks, and hacks for interacting with GenAI tools
• And more!
—
Brought to you by:
• Codeium: Join the 700K+ developers using the IT-approved AI-powered code assistant https://codeium.com/?utm_campaign=pragmatic
• TLDR: Keep up with tech in 5 minutes https://tldr.tech/
Where to find Simon Willison:
• X: https://x.com/simonw
• LinkedIn: https://www.linkedin.com/in/simonwillison/
• Website: https://simonwillison.net/
• Mastodon: https://fedi.simonwillison.net/@simon
Where to find Gergely:
• Newsletter: https://www.pragmaticengineer.com/
• LinkedIn: https://www.linkedin.com/in/gergelyorosz/
• X: https://x.com/GergelyOrosz
In this episode, we cover:
(02:15) Welcome
(05:28) Simon’s ‘scary’ experience with ChatGPT
(10:58) Simon’s initial experiments with LLMs and coding tools
(12:21) The languages that LLMs excel at
(14:50) To start LLMs by understanding the theory, or by playing around?
(16:35) Fine-tuning: what it is, and why it’s mostly a waste of time
(18:03) Where fine-tuning works
(18:31) RAG: an explanation
(21:34) The expense of running testing on AI
(23:15) Simon’s current AI stack
(29:55) Common misconceptions about using LLM tools
(30:09) Simon’s stack – continued
(32:51) Learnings from running local models
(33:56) The impact of Firebug and the introduction of open-source
(39:42) How Simon’s productivity has increased using LLM tools
(41:55) Why most people should limit themselves to 3-4 programming languages
(45:18) Addressing ethical issues and resistance to using generative AI
(49:11) Are LLMs are plateauing? Is AGI overhyped?
(55:45) Coding vs. professional coding, looking ahead
(57:27) The importance of systems thinking for software engineers
(1:01:00) Simon’s advice for experienced engineers
(1:06:29) Rapid-fire questions
(1:09:26) Takeaways
The Pragmatic Engineer deepdives mentioned in this episode:
• Applied AI software engineering: RAG https://newsletter.pragmaticengineer.com/p/rag
• AI tooling for software engineers: reality check https://newsletter.pragmaticengineer.com/p/ai-tooling-2024
See the transcript and other references during the episode at https://newsletter.pragmaticengineer.com/p/ai-tools-for-software-engineers-simon-willison
Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email podcast@pragmaticengineer.com.