Sign in

Preetum Nakkiran

@preetumnakkiran.bsky.social
1.8K followers 1.3K following 37 posts

ML Research @ Apple. Understanding deep learning (generalization, calibration, diffusion, etc). preetum.nakkiran.org

PostsRepliesMedia
Preetum Nakkiran @preetumnakkiran.bsky.social · 25/02/2025
Another intern opening on our team, for a project I’ll be involved in (deadline soon!)
0103
Reposted by Preetum Nakkiran
Chenyang Yuan @chenyang.co · 20/02/2025
Last month I co-taught a class on diffusion models at MIT during the IAP term: www.practical-diffusion.org In the lectures, we first introduced diffusion models from a practitioner's perspective, showing how to build a simple but powerful implementation from the ground up (L1). (1/4)
181
Preetum Nakkiran @preetumnakkiran.bsky.social · 11/02/2025
Paper🧵 (cross-posted at X): When does composition of diffusion models "work"? Intuitively, the reason dog+hat works and dog+horse doesn’t has something to do with independence between the concepts being composed. The tricky part is to formalize exactly what this means. 1/
Left Image: A shaggy dog-horse hybrid standing in a rural landscape.
Right Image: A golden dog wearing a red beret against a blurred outdoor background.
23915
Preetum Nakkiran @preetumnakkiran.bsky.social · 10/02/2025
finally managed to sneak my dog into a paper: arxiv.org/abs/2502.04549
1624
Preetum Nakkiran @preetumnakkiran.bsky.social · 09/02/2025
nice idea actually lol: “Periodic cooking of eggs” : www.nature.com/articles/s44...
172
Reposted by Preetum Nakkiran
Max Puelma Touzel @mptouzel.bsky.social · 05/02/2025
Reminder of a great dictum in research, one of 3 drilled into us by my PhD supervisor: "Don't believe anything obtained only one way", for which the actionable dictum is "immediately do a 2nd independent test of something that looks interesting before in any way betting on it". Its a great activity!
0123
Preetum Nakkiran @preetumnakkiran.bsky.social · 04/02/2025
I’ve been in major denial about how powerful LLMs are, mainly bc I know of no good reason for it to be true. I imagine this was how deep learning felt to theorists the first time around 😬
0220
Reposted by Preetum Nakkiran
Erin Grant @eringrant.me · 21/01/2025
Last year, we funded 250 authors and other contributors to attend #ICLR2024 in Vienna as part of this program. If you or your organization want to directly support contributors this year, please get in touch! Hope to see you in Singapore at #ICLR2025!
13714
Reposted by Preetum Nakkiran
David Pfau @davidpfau.com · 18/01/2025
The thing about "AI progress is hitting a wall" is that AI progress (like most scientific research) is a maze, and the way you solve a maze is by constantly hitting walls and changing directions.
49511
Reposted by Preetum Nakkiran
Pau Rodriguez @paurodriguez.bsky.social · 10/12/2024
Thrilled to share the latest work from our team at @Apple where we achieve interpretable and fine-grained control of LLMs and Diffusion models via Activation Transport 🔥 📄 arxiv.org/abs/2410.23054 🛠️ github.com/apple/ml-act 0/9 🧵
34715
Reposted by Preetum Nakkiran
Brandon Amos @bdamos.bsky.social · 06/01/2025
📢 My team at Meta (including Yaron Lipman and Ricky Chen) is hiring a postdoctoral researcher to help us build the next generation of flow, transport, and diffusion models! Please apply here and message me: www.metacareers.com/jobs/1459691...
metacareers.com
Postdoctoral Researcher, Fundamental AI Research (PhD)
Meta's mission is to build the future of human connection and the technology that makes it possible.
15312
Preetum Nakkiran @preetumnakkiran.bsky.social · 03/01/2025
Giving a short talk at JMM soon, which might finally be the push I needed to learn Lean…
110
Preetum Nakkiran @preetumnakkiran.bsky.social · 01/01/2025
Just read this, neat paper! I really enjoyed Figure 3 illustrating the basic idea: Suppose you train a diffusion model where the denoiser is restricted to be "local" (each pixel i only depends on its 3x3 neighborhood N(i)). The optimal local denoiser for pixel i is E[ x_0[i] | x_t[ N(i) ] ]...cont
1392
Reposted by Preetum Nakkiran
Yoav Goldberg @yoavgo.bsky.social · 26/12/2024
LLMs dont have motives, goals or intents, and so they wont lie or deceive in order to obtain them. but they are fantastic at replicating human culture, and there, goals, intents and deceit abound. so yes, we should also care about such "behaviors" (outputs) in deployed systems.
76210
Reposted by Preetum Nakkiran
Alex Thiery @alexxthiery.bsky.social · 15/12/2024
One #postdoc position is still available at the National University of Singapore (NUS) to work on sampling, high-dimensional data-assimilation, and diffusion/flow models. Applications are open until the end of January. Details: alexxthiery.github.io/jobs/2024_di...
04118
Reposted by Preetum Nakkiran
Eugene Vinitsky 🍒 @eugenevinitsky.bsky.social · 22/12/2024
“Should you still get a PhD given o3” feels like a weird category error. Yes, obviously you should still have fun and learn things in a world with capable AI. What else are you going to do, sit around on your hands?
5886
Preetum Nakkiran @preetumnakkiran.bsky.social · 14/12/2024
Catch our talk about CFG at the M3L workshop Saturday morning @ Neurips! I’ll also be at the morning poster session, happy to chat
1140
Reposted by Preetum Nakkiran
Gautam Kamath @gautamkamath.com · 13/12/2024
Found slides by Ankur Moitra (presented at a TCS For All event) on "How to do theoretical research." Full of great advice! My favourite: "Find the easiest problem you can't solve. The more embarrassing, the better!" Slides: drive.google.com/file/d/15VaT... TCS For all: sigact.org/tcsforall/
313128
Reposted by Preetum Nakkiran
Sander Dieleman @sedielem.bsky.social · 12/12/2024
When a bunch of diffusers sit down and talk shop, their flow cannot be matched😎 It's time for the #NeurIPS2024 diffusion circle! 🕒Join us at 3PM on Friday December 13. We'll meet near this thing, and venture out from there and find a good spot to sit. Tell your friends!
It's located near the west entrance to the west side of the conference center, on the first floor, in case that helps!
1427
Preetum Nakkiran @preetumnakkiran.bsky.social · 06/12/2024
I used to not really care to take pictures of myself, because what would I do with them, but over the break I had a fun training a bunch of Flux LoRAs on myself (surprisingly easy). And now I have renewed interest in profile photos.
130
Reposted by Preetum Nakkiran
Hattie @hattie.bsky.social · 03/12/2024
Help, I'm going to NeurIPS next week! If you are presenting something that you think I should check out, pls drop the deets in thread (1-2 sentence description + where to find you)! 🙏
031
Reposted by Preetum Nakkiran
Sam Power @spmontecarlo.bsky.social · 02/12/2024
An aspect of flow matching which I find a bit interesting is that it is covariant under affine changes of coordinate (c.f. optimal transport, which need not be). This allows for a few nice WLOGs, which I imagine have more applications than I realise.
1223
Reposted by Preetum Nakkiran
ruiqigao.bsky.social @ruiqigao.bsky.social · 02/12/2024
Blog post link: diffusionflow.github.io/ Despite seeming similar, there is some confusion in the community about the exact connection between the two frameworks. We aim to clear up the confusion by showing how to convert one framework to another, for both training and sampling.
diffusionflow.github.io
Diffusion Meets Flow Matching
Flow matching and diffusion models are two popular frameworks in generative modeling. Despite seeming similar, there is some confusion in the community about their exact connection. In this post, we a...
1388
Preetum Nakkiran @preetumnakkiran.bsky.social · 30/11/2024
Sander poses a good question in this thread: Why is there so little variance in learning of diffusion models? Ie, why do different models trained on [random samples from] the same data distribution end up learning *nearly-identical* Noise --> Data function? Some background & speculations:
4434
Reposted by Preetum Nakkiran
Sander Dieleman @sedielem.bsky.social · 28/11/2024
IMO VQGAN is why GANs deserve the NeurIPS test of time award. Suddenly our image representations were an order of magnitude more compact. Absolute game changer for generative modelling at scale, and the basis for latent diffusion models.
arxiv.org
Taming Transformers for High-Resolution Image Synthesis
Designed to learn long-range interactions on sequential data, transformers continue to show state-of-the-art results on a wide variety of tasks. In contrast to CNNs, they contain no inductive bias tha...
210416
Reposted by Preetum Nakkiran
Jon Barron @jonbarron.bsky.social · 25/11/2024
Our group at Google DeepMind is now accepting intern applications for summer 2025. Attached is the official "call for interns" email; the links and email aliases that got lost in the screenshot are below.
39626
Preetum Nakkiran @preetumnakkiran.bsky.social · 23/11/2024
Vague ideas that may not be strictly true are called “intuitions” or “heuristics” or sometimes even “principles” or “laws” (depending on the field). And in the long term such ideas can be extremely impactful.
3142
Reposted by Preetum Nakkiran
Rex "garbage in" Douglass @rexdouglass.bsky.social · 23/11/2024
A list of all the stats/modeling/ML/data starter packs I've seen (26+ and counting):
64412
Preetum Nakkiran @preetumnakkiran.bsky.social · 23/11/2024
(it’s that time of year again..) Wonder if it’d help to make AC names visible to reviewers, and encourage both to keep in touch. That way, outstanding reviewers have an “in” to collab with more senior members of the community — similar to how ugrads get into research by being exceptional in class
170
Preetum Nakkiran @preetumnakkiran.bsky.social · 23/11/2024
despite my best efforts over the years, the biggest factor in my mood is often still “how well is the research going”
1120
Reposted by Preetum Nakkiran
Dylan Foster 🐢 @djfoster.bsky.social · 21/11/2024
As my first post on this platform, allow me to advertise the RL theory lecture notes I have been developing with Sasha Rakhlin: arxiv.org/abs/2312.16730 (shameless repost of my pinned tweet)
421135
Reposted by Preetum Nakkiran
Kira Goldner @kiragoldner.bsky.social · 19/11/2024
I wrote a Part IV postscript to my job market blog post to add what I've learned as faculty. TL;DR: No one is out to get you. For anything not going your way, it's probably due to people being busy or bureaucracy. And there are probably people working very hard for you behind the scenes regardless.
kiragoldner.com
The Job Market (Parts I, II, III, & IV)
0368