Sign in

Quanquan Gu

@quanquangu.bsky.social
1.8K followers 558 following 70 posts

Professor @UCLA, Research Scientist @ByteDance | Recent work: SPIN, SPPO, DPLM 1/2, GPM, MARS | Opinions are my own

PostsRepliesMedia
Reposted by Quanquan Gu
Nathaniel Blalock @nathanielblalock.bsky.social · 20/12/2024
Papers #2-3: arxiv.org/abs/2402.10210 and arxiv.org/abs/2405.00675 from the incredible @quanquangu.bsky.social. I really like how they explore new techniques for RLHF
arxiv.org
Self-Play Fine-Tuning of Diffusion Models for Text-to-Image Generation
Fine-tuning Diffusion Models remains an underexplored frontier in generative artificial intelligence (GenAI), especially when compared with the remarkable progress made in fine-tuning Large Language M...
143
Quanquan Gu @quanquangu.bsky.social · 14/12/2024
Pretraining will only end once we find the optimal scaling law.
170
Quanquan Gu @quanquangu.bsky.social · 05/12/2024
To better interpret the plot, draw a horizontal line representing a specific target validation loss. Find the points where this line intersects the curves for AdamW and MARS, which will allow you to determine how much speedup, in terms of training tokens, MARS achieves compared to AdamW.
000
Quanquan Gu @quanquangu.bsky.social · 03/12/2024
Just added you.
020
Quanquan Gu @quanquangu.bsky.social · 03/12/2024
With the delivery of MARS complete, the focus now shifts to delivering new architectures.
buff.ly
GitHub - AGI-Arena/MARS: The official implementation of MARS: Unleashing the Power of Variance Reduction for Training Large Models
The official implementation of MARS: Unleashing the Power of Variance Reduction for Training Large Models - AGI-Arena/MARS
032
Quanquan Gu @quanquangu.bsky.social · 03/12/2024
Just added you! Welcome!
010
Quanquan Gu @quanquangu.bsky.social · 02/12/2024
Just added you.
000
Quanquan Gu @quanquangu.bsky.social · 01/12/2024
Just added you.
010
Quanquan Gu @quanquangu.bsky.social · 30/11/2024
Just added you!
010
Quanquan Gu @quanquangu.bsky.social · 29/11/2024
Just added you!
110
Quanquan Gu @quanquangu.bsky.social · 29/11/2024
Just added you.
110
Quanquan Gu @quanquangu.bsky.social · 29/11/2024
This Thanksgiving, I want to express my heartfelt gratitude to all the students, colleagues, and collaborators who have contributed to the success of SPIN, SPPO, DPLM, GPM, MARS, and many other projects. Your hard work and dedication continue to be truly inspiring.
0140
Quanquan Gu @quanquangu.bsky.social · 28/11/2024
Just added you!
010
Quanquan Gu @quanquangu.bsky.social · 28/11/2024
Just added you!
010
Quanquan Gu @quanquangu.bsky.social · 28/11/2024
Just added you.
010
Quanquan Gu @quanquangu.bsky.social · 28/11/2024
Anyone using their real name and interested is welcome!
000
Quanquan Gu @quanquangu.bsky.social · 28/11/2024
Just added you. Welcome!
010
Quanquan Gu @quanquangu.bsky.social · 28/11/2024
MARS is a unified framework that can be integrated with various precondition techniques. So it can be applied to PSGD. I believe @hessianfree.bsky.social has implemented MARS-PSGD.
230
Quanquan Gu @quanquangu.bsky.social · 28/11/2024
Just added you!
010
Quanquan Gu @quanquangu.bsky.social · 28/11/2024
Just added you.
110
Quanquan Gu @quanquangu.bsky.social · 28/11/2024
Done!
010
Quanquan Gu @quanquangu.bsky.social · 28/11/2024
Just added you.
000
Quanquan Gu @quanquangu.bsky.social · 28/11/2024
Just added you!
000
Quanquan Gu @quanquangu.bsky.social · 28/11/2024
Just added you!
010
Quanquan Gu @quanquangu.bsky.social · 27/11/2024
Please reply to this message or DM me if you’d like to be added!
330
Quanquan Gu @quanquangu.bsky.social · 27/11/2024
Just added you!
010
Quanquan Gu @quanquangu.bsky.social · 27/11/2024
Have added both of you. Feel free to recommend other people.
010
Reposted by Quanquan Gu
Nathan Lambert @natolambert.bsky.social · 26/11/2024
Tulu 3 SFT mix trending on HuggingFace :D , next step make preferences and RL datasets more accessible.
0152
Reposted by Quanquan Gu
Luca Soldaini 🎀 @soldaini.net · 26/11/2024
OLMo 2 is out 🥳 7B and 13B trained on 5T tokens, and meticulousy instruction tuned using Tulu 3 recipe. Simply the best fully open models yet. Really proud of the work & the amazing team at @ai2.bsky.social
926044
Quanquan Gu @quanquangu.bsky.social · 26/11/2024
Just added you there.
010
Quanquan Gu @quanquangu.bsky.social · 26/11/2024
Just added you. Welcome!
010
Quanquan Gu @quanquangu.bsky.social · 26/11/2024
Just added you. Welcome!
000
Quanquan Gu @quanquangu.bsky.social · 26/11/2024
Just added you!
010
Quanquan Gu @quanquangu.bsky.social · 26/11/2024
Just added you!
010
Reposted by Quanquan Gu
Lenore Blum @lenoreblum.bsky.social · 26/11/2024
Scene thru my window, today, Pittsburgh.
3732
Reposted by Quanquan Gu
Ioannis Kontoyiannis @kontoyiannis.bsky.social · 25/11/2024
Replacing the mean by an arbitrary constant, part 2. This very cool fact is from Tom Cover's 1990 Shannon Lecture: #MathSky
Excerpt of a mathematical statement from Tom Cover's 1990 Shannon Lecture.
1161
Quanquan Gu @quanquangu.bsky.social · 26/11/2024
Just added you. Welcome!
010
Quanquan Gu @quanquangu.bsky.social · 26/11/2024
Just added you.
100
Quanquan Gu @quanquangu.bsky.social · 26/11/2024
Just added you.
010
Reposted by Quanquan Gu
Omead Pooladzandi ✈️ NeurIPS'24 @hessianfree.bsky.social · 26/11/2024
PSGD ❤️ MARS MARS is a new exciting variance reduction technique from @quanquangu.bsky.social 's group which can help stabilize and accelerate your deep learning pipeline. All that is needed is a gradient buffer. Here MARS speeds up the convergence of PSGD ultimately leading to a better solution.
2145
Quanquan Gu @quanquangu.bsky.social · 26/11/2024
Can you put their handles here? Will add them.
320
Quanquan Gu @quanquangu.bsky.social · 26/11/2024
Just added you.
010
Quanquan Gu @quanquangu.bsky.social · 26/11/2024
Just added you.
010
Quanquan Gu @quanquangu.bsky.social · 26/11/2024
Just added you. Welcome!
010
Reposted by Quanquan Gu
Surya Ganguli @suryaganguli.bsky.social · 26/11/2024
Deep learning theory has arrived (on blue sky)!
0315
Reposted by Quanquan Gu
RL Theory Virtual Seminars @rl-theory.bsky.social · 25/11/2024
Tomorrow at 6 PM UTC, Asaf Cassel will talk about "Warm-Up Free Policy Optimization: Improved Regret in Linear Markov Decision Processes".
092
Reposted by Quanquan Gu
RL Theory Virtual Seminars @rl-theory.bsky.social · 25/11/2024
We are on Bluesky as well! We will keep posting on both X and here.
0228
Quanquan Gu @quanquangu.bsky.social · 25/11/2024
Just added you. Welcome!
010
Reposted by Quanquan Gu
Quanquan Gu @quanquangu.bsky.social · 23/11/2024
Just created the Starter Pack for Optimization Researchers to help you on your journey into optimization! 🚀 Did I miss anyone? Tag them or let me know what to add! go.bsky.app/VjpyyRw
13378
Quanquan Gu @quanquangu.bsky.social · 25/11/2024
Just added you!
000