In this special edition, Dr. Tim Scarfe, Yannic Kilcher and Dr. Keith Duggar speak with Professor Gary Marcus, Dr. Walid Saba and Connor Leahy about GPT-3. We have all had a significant amount of time to experiment with GPT-3 and show you demos of it in use and the considerations. Do you think GPT-3 is a step towards AGI? Answer in the comments!
00:00:00 Connor's take on LinkedIn
00:00:47 Show teaser
00:20:02 Tim Introduction
00:26:55 First look at GPT-3, python sorting
00:31:05 Search strategy in LMs
00:38:28 Character analogies and Melanie Mitchell
00:44:27 Substitution cipher
00:47:21 Database prompt
00:53:00 Broader Impact Generation
01:02:47 Gary Marcus Interview (Robust.AI)
01:29:11 Connor Leahy Interview (Eleuther.AI)
01:32:29 Connor -- Tabular data
01:33:41 Connor -- other surprising examples?
01:34:54 Connor -- Is interpolated stuff new?
01:37:43 Connor -- structure of the brain / How GPT works
01:41:21 Connor -- Why cant GPT-3 reason?
01:46:30 Connor -- Missing information problem and ideas on our our brains work
01:54:28 Connor -- Topology of brain/models
01:58:49 Connor -- Hardware lottery / LSTM / Transformer
02:01:41 Connor -- NNs are just matrix program search
02:10:32 Connor -- Google -- information retrieval, the new paradigm, how to extract info from GPT-3, RL controller on top?
02:19:38 Connor -- Database example / "pattern matching is Turing complete"
02:23:55 Connor -- Did gpt3 understand?
02:26:30 Connor -- Are the GOFAI people right?
02:27:40 Walid Saba on GPT-3
02:30:41 Walid -- What is understanding and pattern recognition
02:35:56 Walid -- Chomsky would be happy
02:42:13 Walid -- Redefining success
02:46:05 Walid on Hinton
02:47:34 Walid on software 3.0
02:53:11 Keith -- We use machine learning because we cant write code to do the same thing
02:59:36 Keith -- What is pattern recognition and understanding
03:14:06 GPT-3 trials -- Turing Dialog
03:15:35 GPT-3 trials -- Mary Enjoyed a Sandwich
03:16:19 GPT-3 trials -- BBC has five offices in Germany.
03:16:55 GPT-3 trials -- Database prompt
03:20:23 GPT-3 trials -- Python
03:20:31 GPT-3 trials -- Patterns
03:21:01 GPT-3 trials -- Database again
03:25:11 GPT-3 trials -- GPT-3 experiment -- the trophy doesn’t fit in the suitcase
03:27:32 GPT-3 trials -- Scrambling words
03:30:41 GPT-3 trials -- PDF cleanup example (Gwern)
03:35:03 GPT-3 trials -- Word breaking and simple text patterns
03:37:16 GPT-3 trials -- Typing of entities
03:38:30 GPT-3 trials -- Basic Python append
03:39:07 GPT-3 trials -- Automatic programming?
03:42:31 GPT-3 trials -- Passive aggressive dialog input
03:44:39 GPT-3 trials -- symptoms of depression
03:45:43 GPT-3 trials -- Red shirts reasoning challenge
03:49:59 GPT-3 trials -- Binary encoding
03:50:36 Concluding statements from Walid, Tim and Yannic
Pod version: https://anchor.fm/machinelearningstreettalk/episodes/031-WE-GOT-ACCESS-TO-GPT-3--With-Gary-Marcus--Walid-Saba-and-Connor-Leahy-en2h1k
Connor Leahy:
https://www.linkedin.com/in/connor-j-leahy/
https://twitter.com/NPCollapse
Eleuther.AI Discord -- https://discord.com/invite/vtRgjbM
Gary Marcus:
https://www.linkedin.com/in/gary-marcus-b6384b4/
https://twitter.com/GaryMarcus
https://www.robust.ai
Walid Saba:
https://www.linkedin.com/in/walidsaba/
https://medium.com/ontologik
https://ontologik.ai