Learning to (Learn at Test Time): RNNs with Expressive Hidden States
Video Not Working? Fix It Now
Paper here: https://arxiv.org/abs/2407.04620
Code!: https://github.com/test-time-training/ttt-lm-pytorch
Notes: https://drive.google.com/file/d/127a1UBm_IN_WMKG-DmEvfJ8Pja-9BwDk/view?usp=drive_link
00:00 Intro
04:40 Problem with RNNs
06:38 Meta learning and method idea
09:13 Update rule and RNN inner loop
15:07 Learning the loss function outer loop
21:21 Parallelizing training
30:05 Results
Machine Learning
Artificial Intelligence
Deep Learning
Transformers
Paper Reviews
Text Generation
Generative Models
Text Synthesis
Attention
Self-Attention
Mamba
State Space Models
Selective State Space Models
RNNs
Recurrent Neural Networks
linear attention
Meta learning
learning to learn
Learning to (Learn at Test Time): RNNs with Expressive Hidden States
RNNs with Expressive Hidden States
Comment
