Google TITANS and Sakana Transformer². AI gets memory and can keep learning after training. #ai #ainews #agi
Thanks to our sponsor Abacus AI. Try their ChatLLM platform here: http://chatllm.abacus.ai/?token=aisearch
Titans: Learning to Memorize at Test Time https://arxiv.org/abs/2501.00663
Sakana Transformer² https://sakana.ai/transformer-squared/
0:00 Intro
0:34 Google Titans architecture
1:44 Transformers model and limitations
5:14 Giving AI memory
6:35 AI training vs test time
8:57 Titans long term memory
12:59 Titans architecture
19:50 Pros and cons
24:00 Performance of Titans models vs Transformers
27:31 Sakana Transformer2
30:50 Model design and rationale
32:24 How Transformer2 works
36:00 Transformer2 performance
38:12 Conclusion
Newsletter: https://aisearch.substack.com/
Find AI tools & jobs: https://ai-search.io/
Support: https://ko-fi.com/aisearch
Here's my equipment, in case you're wondering:
Dell Precision 5690: https://www.dell.com/en-us/dt/ai-technologies/index.htm?utm_source=AISearchTools&utm_medium=youtube&utm_campaign=precisionai#tab0=0
GPU: Nvidia RTX 5000 Ada https://nvda.ws/3zfqGqS
Mouse/Keyboard: ALOGIC Echelon https://bit.ly/alogic-echelon
Mic: Shure SM7B https://amzn.to/3DErjt1
Audio interface: Scarlett Solo https://amzn.to/3qELMeu