Sign in

davidgrangier.bsky.social

@davidgrangier.bsky.social
24 followers 23 following 5 posts
PostsRepliesMedia
davidgrangier.bsky.social @davidgrangier.bsky.social · 02/12/2025
I am at #NeurIPS2025. We can chat about data mixing, efficient training, ML@Apple and more.
021
davidgrangier.bsky.social @davidgrangier.bsky.social · 02/12/2025
#NeurIPS2025. Mixing different datasets to train your LLM? ✨ We can help you find the perfect blend! 📈 Few small-model experiments → scaling law fit → your optimal mixture. 🎯 Easy + efficient. Chat with us 💬 Poster #3414. Thu, Dec 4, 11am arxiv.org/abs/2507.09404
arxiv.org
Scaling Laws for Optimal Data Mixtures
Large foundation models are typically trained on data from multiple domains, with the data mixture--the proportion of each domain used--playing a critical role in model performance. The standard appro...
100
davidgrangier.bsky.social @davidgrangier.bsky.social · 22/04/2025
3/3 Mixture of experts on high latency networks with No Need to Talk iclr.cc/virtual/2025... (Thu Apr 24 3pm). Joint work with @matpagliardini.bsky.social , Anastasiia Filippova, @pierreablin.bsky.social, Simin Fan, Skyler Seto, Angelos Katharopoulos, Ronan Collobert
iclr.cc
ICLR Poster No Need to Talk: Asynchronous Mixture of Language ModelsICLR 2025
000
davidgrangier.bsky.social @davidgrangier.bsky.social · 22/04/2025
2/3 Importance sampling for better pretraining distribution with CRISP iclr.cc/virtual/2025... (Sat Apr 26, 10 am).
iclr.cc
ICLR Poster Task-Adaptive Pretrained Language Models via Clustered-Importance SamplingICLR 2025
110
davidgrangier.bsky.social @davidgrangier.bsky.social · 22/04/2025
#ICLR #TrainBetterLM I am at ICLR, come to our posters for improved language model training! Recycle gradients for faster neural net training with AdEMAmix iclr.cc/virtual/2025... (Fri Apr 25, 10 am). 1/3
123