Ai2 @ai2.bsky.social · 05/03/2026Olmo Hybrid’s gains hold across pretraining evals. After pretraining and mid-training, Olmo Hybrid outperforms Olmo 3 in every primary evaluation domain. It wins big on long-context too—on RULER 64k, performance jumps from 70.9% to 85.0%. 🚀 140
Ai2 @ai2.bsky.social · 05/03/2026What explains the success of Olmo Hybrid? We prove hybrid models are more expressive than transformers or RNNs alone, being capable of representing a broader class of functions related to code evaluation. 140
Ai2 @ai2.bsky.social · 05/03/2026Moreover, we give a theoretical argument tying this greater expressive power to the improved scaling of hybrid models we see in practice. 🧑🏫 130
