iiinova — where your digital self keeps evolving

Reposted by Yuda Song

Miro Dudik @mdudik.bsky.social · 18/09/2025

🚨Microsoft Research NYC is hiring🚨 We're hiring postdocs and senior researchers in AI/ML broadly, and in specific areas like test-time scaling and science of DL. Postdoc applications due Oct 22, 2025. Senior researcher applications considered on a rolling basis. Links to apply: aka.ms/msrnyc-jobs

aka.ms

Microsoft Research Lab - New York City - Microsoft Research

Apply for a research position at Microsoft Research New York & collaborate with academia to advance economics research, prediction markets & ML.

0187

Reposted by Yuda Song

Jacob Springer @jacobspringer.bsky.social · 26/03/2025

Training with more data = better LLMs, right? 🚨 False! Scaling language models by adding more pre-training data can decrease your performance after post-training! Introducing "catastrophic overtraining." 🥁🧵👇 arxiv.org/abs/2503.19206 1/10

13314

Reposted by Yuda Song

Gokul Swamy @gokul.dev · 04/03/2025

1.5 yrs ago, we set out to answer a seemingly simple question: what are we *actually* getting out of RL in fine-tuning? I'm thrilled to share a pearl we found on the deepest dive of my PhD: the value of RL in RLHF seems to come from *generation-verification gaps*. Get ready to 🤿:

15911

Reposted by Yuda Song

Antoine Moulin @antoine-mln.bsky.social · 20/02/2025

super happy about this preprint! we can *finally* perform efficient exploration and find near-optimal stationary policies in infinite-horizon linear MDPs, and even use it for imitation learning :) working with @neu-rips.bsky.social and @lviano.bsky.social on this was so much fun!!

2232

Reposted by Yuda Song

Dylan Foster 🐢 @djfoster.bsky.social · 20/02/2025

What are the minimal supervised learning primitives required to perform RL efficiently? New paper led by my amazing intern Dhruv Rohatgi: Necessary and Sufficient Oracles: Toward a Computational Taxonomy for Reinforcement Learning arxiv.org/abs/2502.08632 1/

1254

Reposted by Yuda Song

Leshem (Legend) Choshen @EMNLP @lchoshen.bsky.social · 13/12/2024

Models can self-improve🥷 by knowing they were wrong🧘‍♀️ but when can they do it? Across LLM families, tasks and mechanisms This ability scales with pretraining, prefers CoT, non QA tasks and more in 🧵 alphaxiv.org/abs/2412.02674 @yus167.bsky.social @shamkakade.bsky.social 📈🤖 #NLP #ML

alphaxiv.org

Mind the Gap: Examining the Self-Improvement Capabilities of Large Language Models | alphaXiv

View 3 comments: Delete the space?

2233

Yuda Song @yus167.bsky.social · 09/12/2024

I will present two papers at #NeurIPS2024! Happy to meet old and new friends and talk about all aspects of RL: data, environment structure, and reward! 😀 In Wed 11am-2pm poster session I will present HyPO-- best of both worlds of offline and online RLHF: neurips.cc/virtual/2024...

neurips.cc

NeurIPS Poster The Importance of Online Data: Understanding Preference Fine-tuning via CoverageNeurIPS 2024

192

Yuda Song @yus167.bsky.social · 06/12/2024

LLM self-improvement has critical implications in synthetic data, post-training and test-time inference. To understand LLMs' true capability of self-improvement, we perform large-scale experiments with multiple families of LLMs, tasks and mechanisms. Here is what we found: (1/9)

1124

Reposted by Yuda Song

arxiv cs.CL @arxiv-cs-cl.bsky.social · 04/12/2024

Yuda Song, Hanlin Zhang, Carson Eisenach, Sham Kakade, Dean Foster, Udaya Ghai Mind the Gap: Examining the Self-Improvement Capabilities of Large Language Models arxiv.org/abs/2412.02674

011

Reposted by Yuda Song

Gokul Swamy @gokul.dev · 22/11/2024

I think the main difference in terms of interpolation / extrapolation between DPO and RLHF is that the former only guarantees closeness to the reference policy on the training data, while RLHF usually tacks on an on-policy KL penalty. We explored this point in arxiv.org/abs/2406.01462.

141

Reposted by Yuda Song

Sham Kakade @shamkakade.bsky.social · 22/11/2024

(1/n) 💡How can we speed up the serial runtime of long pre-training runs? Enter Critical Batch Size (CBS): the tipping point where the gains of data parallelism balance with diminishing efficiency. Doubling batch size halves the optimization steps—until we hit CBS, beyond which returns diminish.

2174

Reposted by Yuda Song

steph milani @stephmilani.bsky.social · 18/11/2024

I created a starter pack for people who are or have been affiliated with the Machine Learning Department at CMU. Let me know if I missed someone! go.bsky.app/QLTVEph #AcademicSky

044

Reposted by Yuda Song

arxiv stat.ML @arxiv-stat-ml.bsky.social · 22/11/2024

Ojash Neopane, Aaditya Ramdas, Aarti Singh Logarithmic Neyman Regret for Adaptive Estimation of the Average Treatment Effect arxiv.org/abs/2411.14341

051

Reposted by Yuda Song

Zhengyi "Zen" Luo @zhengyiluo.bsky.social · 19/11/2024

Intro 🦋 I am a final-year PhD student from CMU Robotics. I work on humanoid control, perception, and behavior in both simulation and real life, using mostly RL: 🏃🏻PHC: zhengyiluo.com/PHC 💫PULSE: zhengyiluo.com/PULSE 🔩Omnigrasp: zhengyiluo.com/Omnigrasp 🤖OmniH2O: omni.human2humanoid.com

2223

Reposted by Yuda Song

steph milani @stephmilani.bsky.social · 18/11/2024

Hi Bsky people 👋 I'm a PhD candidate in Machine Learning at Carnegie Mellon University. My research focuses on interactive AI, involving: 🤖 reinforcement learning, 🧠 foundation models, and 👩‍💻 human-centered AI. Also a founding co-organizer of the MineRL competitions 🖤 Follow me for ML updates!

2706