This presentation was recorded at GOTO Amsterdam 2024. #GOTOcon #GOTOams
https://gotoams.nl
Alan Smith - AI Developer, Trainer, Mentor & Evangelist at Active Solution @CloudCastsAlanSmith
RESOURCES
https://twitter.com/alansmith
https://www.linkedin.com/in/alan-smith-68a8491
ABSTRACT
Natural language processing using generative pre-trained transformers (GPT) algorithms is a rapidly evolving field that offers many opportunities and challenges for application developers. But what is a generative pre-trained transformer, and how does it work? How can you leverage the latest advances in GPT algorithms to create engaging and useful applications? Can my business benefit from creating a GPT powered chat bot?
In this demo intensive session Alan will take a deep dive into the architecture of GPT algorithms and the inner workings of ChatGPT. The journey will begin by looking at the fundamental concepts of natural language processing, such as word embedding, vectorization and tokenization. He will then demonstrate how you can apply these techniques to train a GPT2 model that can generate song lyrics, showing the internals of how word sequences are predicted.
Alan will then shift the focus to larger language models, such as ChatGPT and GPT4, demonstrating their power, capabilities, and limitations. The use of hyperparameters such as temperature and frequency penalty will be explained and their effect on the generated output demonstrated.
Join me for this session if you want to learn how to harness the power of GPT algorithms in your own solutions. [...]
TIMECODES
00:00 Intro
01:59 GPT sequence prediction
03:48 Prompt engineering
04:45 Demo: ChatGPT2
10:45 Processing text
19:27 Demo: Word2Vec dimensionality reduction
21:07 Transformer architecture
23:58 Demo: GPT2 input embedding
27:00 Self attention
31:08 Demo: GPT2 multi-head attention
32:52 Attention example
33:49 Demo: GPT2 next token prediction
44:06 Parameters
46:21 Thanks for explanations & inspiration
46:57 Outro
Read the full abstract here:
https://gotoams.nl/2024/sessions/3145
RECOMMENDED BOOKS
Alex Castrounis • AI for People and Business • https://amzn.to/3NYKKTo
Phil Winder • Reinforcement Learning • https://amzn.to/3t1S1VZ
Holden Karau, Trevor Grant, Boris Lublinsky, Richard Liu & Ilan Filonenko • Kubeflow for Machine Learning • https://amzn.to/3JVngcx
Kelleher & Tierney • Data Science (The MIT Press Essential Knowledge series) • https://amzn.to/3AQmIRg
Lakshmanan, Robinson & Munn • Machine Learning Design Patterns • https://amzn.to/2ZD7t0x
Lakshmanan, Görner & Gillard • Practical Machine Learning for Computer Vision • https://amzn.to/3m9HNjP
https://twitter.com/GOTOcon
https://www.linkedin.com/company/goto-
https://www.instagram.com/goto_con
https://www.facebook.com/GOTOConferences
#LLM #LargeLanguageModel #LimitationsOfLLMs #InsideGPT #GPT #GPT4 #GPT2 #GenAI #AI #GenerativeAI #CUDA #CommonCrawl #LSTMs #AlanSmith
CHANNEL MEMBERSHIP BONUS
Join this channel to get early access to videos & other perks:
https://www.youtube.com/channel/UCs_tLP3AiwYKwdUHpltJPuA/join
Looking for a unique learning experience?
Attend the next GOTO conference near you! Get your ticket at https://gotopia.tech
Sign up for updates and specials at https://gotopia.tech/newsletter
SUBSCRIBE TO OUR CHANNEL - new videos posted almost daily.
https://www.youtube.com/user/GotoConferences/?sub_confirmation=1