Ai2 @ai2.bsky.social · 01/10/2026Olmo-core 2 repeatedly gathered model weights for each small batch of training data. Olmo-core 3 keeps experts resident on GPUs & sends data to them, avoiding that repeated weight gathering. 180 0
Ai2 @ai2.bsky.social · 01/10/2026Olmo-core 3 maintains nearly the same training speed across MoEs of increasing total size, w/ roughly the same number of parameters active per token. It delivers about 2.7× the throughput of Olmo-core 2 for the same model on the same number of GPUs. 180 0
Ai2 @ai2.bsky.social · 01/10/2026Explore the concepts behind Olmo-core 3: buff.ly/kbV4bQu The tech report covers experiments, findings, & failures, including token gerrymandering, where a load-balancing score could improve as workloads became less balanced: buff.ly/I5Z2Q4F 290 0
