Sign in

Gokul Swamy

@gokul.dev
4.4K followers 417 following 109 posts

PhD student at @cmurobotics.bsky.social working on efficient algorithms for interactive learning (e.g. imitation / RL / RLHF). no model is an island. prefers email. gokul.dev. on the job market!

PostsRepliesMedia
Gokul Swamy @gokul.dev · 26/11/2025
Couldn't think of a better excuse than #neurips2025 to visit my home town of San Diego! I'll be around 12/3-12/6 -- shoot me a DM / email if you'd like to catch up!
010
Reposted by Gokul Swamy
Natalie Collina @ncollina.bsky.social · 22/10/2025
Our paper on algorithmic collusion was featured in a Quanta article! www.quantamagazine.org/the-game-the...
quantamagazine.org
The Game Theory of How Algorithms Can Drive Up Prices | Quanta Magazine
Recent findings reveal that even simple pricing algorithms can make things more expensive.
2309
Gokul Swamy @gokul.dev · 22/10/2025
Just discovered this lovely talk from my favorite professor from undergrad, who is still as inspiring as I remember him: www.youtube.com/watch?v=XLZ0...
youtube.com
“Raising Our Sights (A Long Rant From an Accidental Engineer)”, Scott Shenker
YouTube video by Plamadiso - Platforms, Markets & Digital Society
000
Gokul Swamy @gokul.dev · 13/10/2025
Please apply or help out!
010
Gokul Swamy @gokul.dev · 28/09/2025
Late, but arxiv.org/abs/0804.2996 is *incredible*, so many good lines (e.g., "This comes close to being an accusation of a false claim of priority for a false discovery of an untrue fact, which would be a rare triple-negative in the history of intellectual property disputes.").
arxiv.org
The Epic Story of Maximum Likelihood
At a superficial level, the idea of maximum likelihood must be prehistoric: early hunters and gatherers may not have used the words ``method of maximum likelihood'' to describe their choice of where a...
0113
Gokul Swamy @gokul.dev · 15/07/2025
Recent work has seemed somewhat magical: how can RL with *random* rewards make LLMs reason? We pull back the curtain on these claims and find out this unexpected behavior hinges on the inclusion of certain *heuristics* in the RL algorithm. Our blog post: tinyurl.com/heuristics-c...
tinyurl.com
Heuristics Considered Harmful: RL With Random Rewards Should Not Make LLMs Reason | Notion
Owen Oertell*, Wenhao Zhao*, Gokul Swamy, Zhiwei Steven Wu, Kiante Brantley, Jason Lee, Wen Sun
162
Reposted by Gokul Swamy
quokkka.bsky.social @quokkka.bsky.social · 20/06/2025
very nice lectures, watch them from time to time
061
Reposted by Gokul Swamy
Marc Lanctot @sharky6000.bsky.social · 20/06/2025
Want to learn about online learning, game solving, RL, imitation learning with applications to robotics, and RLHF with applications to language modeling? Check out this course! 👍
061
Gokul Swamy @gokul.dev · 20/06/2025
It was a dream come true to teach the course I wish existed at the start of my PhD. We built up the algorithmic foundations of modern-day RL, imitation learning, and RLHF, going deeper than the usual "grab bag of tricks". All 25 lectures + 150 pages of notes are now public!
34710
Gokul Swamy @gokul.dev · 13/06/2025
Shortcut models enable scaling offline RL, both at train-time at test-time! We beat so many other algorithms on so many tasks we had to stick most of the results in the appendix 😅. Very proud of @nico-espinosa-dice.bsky.social for spearheading this project, check out his thread!
011
Gokul Swamy @gokul.dev · 27/04/2025
Boston friends: I'll be in the Cambridge area for the next few days, shoot me a message if you'd like to catch up :).
010
Gokul Swamy @gokul.dev · 22/04/2025
I won't be at #ICLR2025 myself this time around but please go talk to lead authors Nico, Zhaolin, and Runzhe about their bleeding-edge algorithms for imitation learning and RLHF!
030
Gokul Swamy @gokul.dev · 07/04/2025
I think of misspecification (e.g. embodiment / sensory gaps) as the fundamental reason behavioral cloning isn't "all you need" for imitation as matching actions matching outcomes. Introducing @nico-espinosa-dice.bsky.social's #ICLR2025 paper proving that "local search" *is* all you need! [1/n]
140
Gokul Swamy @gokul.dev · 06/03/2025
I was lucky enough to be invited give a talk on our new paper on the value of RL in fine-tuning at Cornell last week! Because of my poor time management skills, the talk isn't as polished as I'd like, but I think the "vibes" are accurate enough to share: youtu.be/E4b3cSirpsg.
youtu.be
All Roads Lead to Likelihood: The Value of RL in Fine-Tuning
YouTube video by Gokul Swamy
0153
Gokul Swamy @gokul.dev · 04/03/2025
1.5 yrs ago, we set out to answer a seemingly simple question: what are we *actually* getting out of RL in fine-tuning? I'm thrilled to share a pearl we found on the deepest dive of my PhD: the value of RL in RLHF seems to come from *generation-verification gaps*. Get ready to 🤿:
15911
Gokul Swamy @gokul.dev · 23/02/2025
new h-index final boss just dropped 😭😭😭: scholar.google.com/citations?hl...
050
Reposted by Gokul Swamy
Eugene Vinitsky 🍒 @eugenevinitsky.bsky.social · 18/02/2025
A large group of us (spearheaded by Denizalp Goktas) have put out a position paper on paths towards foundation models for strategic decision-making. Language models still lack these capabilities so we'll need to build them: hal.science/hal-04925309...
2337
Gokul Swamy @gokul.dev · 22/01/2025
3/3 for #ICLR2025! Huge congratulations to lead authors Nico Espinosa-Dice, Zhaolin Gao, Runzhe Wu -- all students of the wonderful Wen Sun! I'll save a more in depth discussion of the papers for later but if you'd like a sneak peak, check out arxiv.org/abs/2410.13855 and arxiv.org/abs/2410.04612!
arxiv.org
Diffusing States and Matching Scores: A New Framework for Imitation Learning
Adversarial Imitation Learning is traditionally framed as a two-player zero-sum game between a learner and an adversarially chosen cost function, and can therefore be thought of as the sequential gene...
070
Reposted by Gokul Swamy
Eugene Vinitsky 🍒 @eugenevinitsky.bsky.social · 20/01/2025
My actual favorite niche starter pack here: go.bsky.app/2Gibu1a
53810
Gokul Swamy @gokul.dev · 16/01/2025
Lynch was perhaps my favorite director: it was like he lifted Murakami to the silver screen. His work had enough surrealism that his message stuck, but not so much that it was hidden. To quote my favorite Twin Peaks episode: "this is the water and this is the well, drink full and descend." RIP.
variety.com
David Lynch, Visionary Director of ‘Twin Peaks’ and ‘Blue Velvet,’ Dies at 78
Director David Lynch, who radicalized American film with with a dark, surrealistic artistic vision in films like 'Blue Velvet,' has died. He was 78.
071
Gokul Swamy @gokul.dev · 02/01/2025
Congrats to my ever-amazing undergrad Juntao Ren (who is far too productive to be on here) for being named a Runner Up for the 2025 CRA Outstanding Undergraduate Researcher Award (cra.org/about/awards...)! He's on the PhD Job Market this year and I can't say enough good things about him!
1120
Gokul Swamy @gokul.dev · 11/12/2024
The more time I spend on RLHF, the more I realize the devil is in the details (even more than RL for continuous control). My co-author Zhaolin Gao wrote this excellent blog post on some of these details: huggingface.co/blog/GitBag/.... Maybe it'll be your savior!
huggingface.co
RLHF 101: A Technical Dive into RLHF
A Blog post by Zhaolin Gao on Hugging Face
0224
Reposted by Gokul Swamy
Reinforcement Learning Conference @rl-conference.bsky.social · 10/12/2024
If you're at NeurIPS, RLC is hosting an RL event from 8 till late at The Pearl on Dec. 11th. Join us, meet all the RL researchers, and spread the word!
26318
Reposted by Gokul Swamy
Yuda Song @yus167.bsky.social · 09/12/2024
I will present two papers at #NeurIPS2024! Happy to meet old and new friends and talk about all aspects of RL: data, environment structure, and reward! 😀 In Wed 11am-2pm poster session I will present HyPO-- best of both worlds of offline and online RLHF: neurips.cc/virtual/2024...
neurips.cc
NeurIPS Poster The Importance of Online Data: Understanding Preference Fine-tuning via CoverageNeurIPS 2024
192
Reposted by Gokul Swamy
Mononito Goswami @mononitogoswami.bsky.social · 09/12/2024
Excited to be at NeurIPS'24, where I'll be presenting at several workshops! Looking forward to chatting about (time series & tabular) foundation models, data science agents, or ML for healthcare! Also, I'm also on the industry job market, looking forward to connect 😁!
0225
Gokul Swamy @gokul.dev · 08/12/2024
I won't be at #NeurIPS2024 this year but some brilliant folks I work with will be presenting some really cool work on imitation learning and RLHF. Please stop by and check out their posters! Quick links to papers: arxiv.org/abs/2406.04219, arxiv.org/abs/2406.01462, arxiv.org/abs/2404.16767.
1112
Reposted by Gokul Swamy
Yuda Song @yus167.bsky.social · 06/12/2024
LLM self-improvement has critical implications in synthetic data, post-training and test-time inference. To understand LLMs' true capability of self-improvement, we perform large-scale experiments with multiple families of LLMs, tasks and mechanisms. Here is what we found: (1/9)
1124
Gokul Swamy @gokul.dev · 04/12/2024
This is a good post (the birds don't hurt either 🦜). Lately, I've been thinking a lot about how a similar principle applies in modern ML research (even in academia). I'm still not sure if this is a good thing, but it is the world we live in.
040
Reposted by Gokul Swamy
Marc Lanctot @sharky6000.bsky.social · 21/11/2024
For more on SPO and related papers, see @gswamy.bsky.social 's thread posted yesterday! bsky.app/profile/gswa...
121
Reposted by Gokul Swamy
Marc Lanctot @sharky6000.bsky.social · 21/11/2024
@gswamy.bsky.social et al propose SPO which builds a game from a preferences, solving for the minimax winner. Handles non-Markovian, intransitive, and stochastic preferences. Nice empirical eval ranging from small demonstrative domains to huge RL domain (Mujoco). arxiv.org/abs/2401.04056 2/3.
arxiv.org
A Minimaximalist Approach to Reinforcement Learning from Human Feedback
We present Self-Play Preference Optimization (SPO), an algorithm for reinforcement learning from human feedback. Our approach is minimalist in that it does not require training a reward model nor unst...
2172
Gokul Swamy @gokul.dev · 12/11/2024
A dream I've had for five years is finally coming true: I'll be co-teaching a course next sem. on the algorithmic foundations of imitation learning / RLHF with my advisors, Drew Bagnell and Steven Wu! Sign up if you're at CMU (17-740) or follow along at interactive-learning-algos.github.io!
interactive-learning-algos.github.io
Home
Website for AFIL course.
0112