2024 in Post-Transformer Architectures: State Space Models, RWKV [Latent Space LIVE! @ NeurIPS 2024]

https://latent.space/p/2024-post-transformers

Eugene Cheah of Recursal/Featherless and Dan Fu of Together bring a concise 30 mins updating all the work on RWKV and State Space Models this year, and riff on some hot takes on long context vs RAG