iiinova — where your digital self keeps evolving

Lots more in the paper: how does DPO fit into the picture? What if attackers have different goals? etc. Paper: arxiv.org/abs/2606.04929 Code: github.com/jcksanderson... Led by Jack Sanderson (jcksanderson.com), w/ Yihan Wang, Xiaoqian Lu, co-supervised w/ Yiwei Lu

010

So far, it seems like the system is shockingly robust, right? Unfortunately, this is an illusion. First row reconfirms [RT24]: with no SFT poison, it's very hard to poison the PPO'd model (needs >5% poison). But, with just a little SFT poison (0.5%), attack succeeds w/ 3%. 5/n

210

What about poisoning PPO? A remarkable paper of @javirandor.com and @floriantramer.bsky.social (arxiv.org/abs/2311.14455) shows that just 0.5% poison is enough to break a reward model (L)! Again, fear not: somehow, it takes a (high) 5% poisoning before it transfers to the RLHF'd model (R). 4/n

210

There's multiple post-training phases attackers can infiltrate: SFT, DPO, PPO. Let's start with SFT. With just 2% SFT poisoning, 90% attack success (L)! But not to worry, RLHF works as we hope (?): it wipes away the poison. An RM scores outputs just like a clean model (R). 3/n

130

Gautam Kamath @gautamkamath.com · 04/06/2026

🧵Feeling safe against data poisoning in post-training? Think again! Individual components of LLM post-training pipelines are surprisingly robust to data poisoning attacks. In work led by Jack Sanderson (co-advised w Yiwei Lu), we show they crumble when attacked together. 1/n

162

Laudatio: sigact.org/prizes/g%C3%... Article by @uwcheritoncs.bsky.social comms guy Joe Petrik. Pic's a few years old (you can tell by the gray hairs). He joked, this shot is for when you win the Turing or the Gödel. Didn't think we'd actually get to use it. 7/7 cs.uwaterloo.ca/news/gautam-...

050

Gautam Kamath @gautamkamath.com · 04/06/2026

Honoured that our 2016 paper, Robust Estimators in High Dimensions without the Computational Intractability, w/ Ilias Diakonikolas, @daneelssoul.bsky.social, @jerryzli.bsky.social, Ankur Moitra, Alistair Stewart, was awarded the Gödel Prize This is the highest award for papers in theoretical CS 1/7

5486

Gautam Kamath @gautamkamath.com · 03/06/2026

"People submit too many papers to ML conferences" Meanwhile, at IEEE Transactions on Wireless Communications: "You may not submit more than 36 papers per year." Apparently this is a new policy. I would be super curious to see the stats on how many people are submitting more.

2142

Gautam Kamath @gautamkamath.com · 03/06/2026

Key tweaks to improve Gemini

050

Check out the full website and submit here! re-data-colm2026.github.io Co-organized with a great bunch including lead organizer Anil Ramakrishna, as well as Zheng Xu, Kamalika Chaudhuri, Om Thakkar, Natalia Ponomareva, and Sasha Korolova.

010

Featuring a great lineup of invited speakers, including @sewonm.bsky.social @alexdimakis.bsky.social @nouhadziri.bsky.social Niloofar Mireshghallah, Janel Thamkul (and maybe more...?)

130

Soliciting papers on data transformation, synthetic data, new training paradigms, evaluation, auditing, policy, and more! We need more PC members! Please sign up here?🥺 docs.google.com/forms/d/e/1F...

100

Workshop on Responsibly Enabling Data for Foundation Models at #COLM2026 October 9 in SF "Unlocking sensitive data sources responsibly for the next generation of AI" - Amazing invited speakers 😍 - Submission deadline: June 23 🗓️ - Do *you* want to be a PC member? 👈 @colmweb.org

183

It's time for the Theory Jobs spreadsheet for 2026! The theory CS community crowdsources who accepted which jobs every year. Blog post: kamathematics.wordpress.com/2026/06/01/t... Form: docs.google.com/forms/d/1Qcn... Results: docs.google.com/spreadsheets...

0186

Paper: peihanliu.com/assets/paper... Blog post: peihanliu.com/posts/contin... HuggingFace: hf.co/ContinuousBe... Huge credit to Peihan Liu and Alex Bie who led this project! 8/8

130

These tasks are hard/impossible to zero-shot, rather easy without privacy, but surprisingly hard even with large privacy budgets (ε = 100)! This room to grow means we can really measure progress made by new DP synthetic data benchmarks. 6/n

100