Ai2 @ai2.bsky.social · 05/03/2026What explains the success of Olmo Hybrid? We prove hybrid models are more expressive than transformers or RNNs alone, being capable of representing a broader class of functions related to code evaluation. 140 0
Ai2 @ai2.bsky.social · 05/03/2026Moreover, we give a theoretical argument tying this greater expressive power to the improved scaling of hybrid models we see in practice. 🧑🏫 130 0
Ai2 @ai2.bsky.social · 05/03/2026Overall, our results suggest compelling advantages for hybrid models over transformers—both theoretically, in terms of expressive power and scaling efficiency, and practically, in terms of benchmark performance and long-context abilities. 150 0
