The Large Concept Model (LCM) shifts from token-based processing to reasoning at the sentence level by embedding sentences as vectors in a high-dimensional SONAR space, enabling multi-lingual and multi-modal generalization. No more predict the next token!
See also this next video on Byte Latent Transformer architecture - BLT:
https://m.youtube.com/watch?v=KZfGgmtQFh0
Large Concept Models - LCM by @meta @MetaDevelopers
All rights w authors:
Large Concept Models
Loïc Barrault, Paul-Ambroise Duquenne, Maha Elbayad, Artyom
Kozhevnikov, Belen Alastruey, Pierre Andrews, Mariano Coria, Guillaume Couairon, Marta R. Costa-jussà, David Dale, Hady Elsahar, Kevin Heffernan, João Maria Janeiro, Tuan Tran, Christophe Ropers, Eduardo Sánchez, Robin San Roman, Alexandre Mourachk, Safiyyah Saleem, Holger Schwenk
#reasoning
#abstraction
#languages
#coding
#airesearch
#meta