The 10 billion parameter SEER model from @MetaAI is *fairer*, even though it is trained on *uncurated* data. How so? Check out our take on this.
SPONSOR: Diffgram 👉 https://diffgram.com/
Check out our daily #MachineLearning Quiz Questions:
❓ https://www.youtube.com/c/AICoffeeBreak/community
➡️ AI Coffee Break Merch! 🛍️ https://aicoffeebreak.creator-spring.com/
Thanks to our Patrons who support us in Tier 2, 3, 4: 🙏
Don Rosenthal, Dres. Trost GbR, banana.dev -- Kyle Morris, Julián Salazar, Edvard Grødem, Vignesh Valliappan
Paper 📜: Goyal, Priya, Quentin Duval, Isaac Seessel, Mathilde Caron, Mannat Singh, Ishan Misra, Levent Sagun, Armand Joulin, and Piotr Bojanowski. "Vision models are more robust and fair when pretrained on uncurated images without supervision." arXiv preprint arXiv:2202.08360 (2022). https://arxiv.org/abs/2202.08360
📜 RegNet paper: https://arxiv.org/abs/2101.00590
📜 SwAV paper: https://arxiv.org/abs/2006.09882
🔗 Official implementation: https://github.com/facebookresearch/vissl/tree/main/projects/SEER
🔗 GPT-3 trained on a curated dataset called “InstructGPT”: https://openai.com/blog/improving-language-model-behavior/
🔗 GPT-3 examples of toxic behaviour: https://venturebeat.com/2022/01/27/openai-rolls-out-new-text-generating-models-that-it-claims-are-less-toxic/
Outline:
00:00 Training on uncurated data
01:12 Diffgram (Sponsor)
01:46 Toxicity in large models
02:43 What to do against model toxicity?
03:53 SEER model explained
06:52 SEER is fairer. But how?
▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
🔥 Optionally, pay us a coffee to help with our Coffee Bean production! ☕
Patreon: https://www.patreon.com/AICoffeeBreak
Ko-fi: https://ko-fi.com/aicoffeebreak
▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
🔗 Links:
AICoffeeBreakQuiz: https://www.youtube.com/c/AICoffeeBreak/community
Twitter: https://twitter.com/AICoffeeBreak
Reddit: https://www.reddit.com/r/AICoffeeBreak/
YouTube: https://www.youtube.com/AICoffeeBreak
#AICoffeeBreak #MsCoffeeBean #MachineLearning #AI #research
Music 🎵 : The Itch (Instrumental) - NEFFEX