Sign in

Michael Poli

@michaelpoli.bsky.social
120 followers 22 following 6 posts

AI, numerics and systems. Founding Scientist at Liquid AI

PostsRepliesMedia
Michael Poli @michaelpoli.bsky.social · 14/11/2024
[6] It's now undeniable that, with a little bit of creativity, improving scaling is not only approachable but also particularly rewarding. And while I'm obviously excited by convolution-attention hyena hybrids due to their balance of efficiency and quality across domains, there's a lot more to do!
000
Michael Poli @michaelpoli.bsky.social · 14/11/2024
[5] We have seen time and again that various classes of computational units outperform others in different modalities, on different tasks, in different regimes. We've seen this in scaling laws, on synthetics, on inference.
100
Michael Poli @michaelpoli.bsky.social · 14/11/2024
[4] There has been a flurry of work over the last couple of years (from the great people at HazyResearch and elsewhere) on developing bespoke model designs as "proofs of existence" to challenge the Transformer orthodoxy, at a time when model design was considered "partially solved."
100
Michael Poli @michaelpoli.bsky.social · 14/11/2024
[3] We continue to push the scale of what's possible with "beyond Transformer" models applied to biology, in what could be among the most computationally intensive fully open (weights, data, pretraining infrastructure) sets of pretrained models across AI as a whole.
100
Michael Poli @michaelpoli.bsky.social · 14/11/2024
[2] A lot has happened since the first release of Evo. We have made public the original pretraining dataset (OpenGenome)—links below—and will soon release the entire pretraining infrastructure and model code.
100
Michael Poli @michaelpoli.bsky.social · 14/11/2024
An absolute privilege to see our work on Evo 🧬 highlighted on the cover of the latest issue of Science. Thank you to all the friends and collaborators at Stanford and the Arc Institute.
220