Sign in

Prasanna Mayilvahanan

@prasannamayil.bsky.social
27 followers 35 following 8 posts

PhD student in ML at MPI-IS. Prev Apple. Interested in robustness at scale and reasoning.

PostsRepliesMedia
Reposted by Prasanna Mayilvahanan
Andreas Hochlehnert @ahochlehnert.bsky.social · 17/02/2025
CuratedThoughts: Data Curation for RL Datasets 🚀 Since DeepSeek-R1 introduced reasoning-based RL, datasets like Open-R1 & OpenThoughts emerged for fine-tuning & GRPO. Our deep dive found major flaws — 25% of OpenThoughts needed elimination by data curation. Here's why 👇🧵
1139
Prasanna Mayilvahanan @prasannamayil.bsky.social · 18/02/2025
New preprint out! 🎉 How does LLM training loss translate to downstream performance? We show that pretraining data and tokenizer shape loss-to-loss scaling, while architecture and other factors play a surprisingly minor role! brendel-group.github.io/llm-line/ 🧵1/8
1188