iiinova — where your digital self keeps evolving

Pete Shaw

@ptshaw.bsky.social

1.6K followers 356 following 9 posts

Research Scientist at Google DeepMind. Mostly work on ML, NLP, and BioML. Based in Seattle. ptshaw.com

Pete Shaw @ptshaw.bsky.social · 01/10/2025

Excited to share a new paper that aims to narrow the conceptual gap between the idealized notion of Kolmogorov complexity and practical complexity measures for neural networks.

Bridging Kolmogorov Complexity and Deep Learning: Asymptotically Optimal Description Length Objectives for Transformers

195

Reposted by Pete Shaw

Ahmad Beirami @abeirami.bsky.social · 01/01/2025

Excited to share 𝐈𝐧𝐟𝐀𝐥𝐢𝐠𝐧! Alignment optimization objective implicitly assumes 𝘴𝘢𝘮𝘱𝘭𝘪𝘯𝘨 from the resulting aligned model. But we are increasingly using different and sometimes sophisticated inference-time compute algorithms. How to resolve this discrepancy?🧵

InfAlign: Inference-aware language model alignment
Ananth Balashankar, Ziteng Sun, Jonathan Berant, Jacob Eisenstein, Michael Collins, Adrian Hutter, Jong Lee, Chirag Nagpal, Flavien Prost, Aradhana Sinha, Ananda Theertha Suresh, Ahmad Beirami

25511

Pete Shaw @ptshaw.bsky.social · 09/12/2024

I'll be at NeurIPS this week. Please reach out if you would like to chat!

160

Reposted by Pete Shaw

Marc Lanctot @sharky6000.bsky.social · 28/10/2024

New starter pack! go.bsky.app/GZ4hZzu

64217

Reposted by Pete Shaw

Kevin K. Yang 楊凱筌 @kevinkaichuang.bsky.social · 18/11/2024

Two BioML starter packs now: Pack 1: go.bsky.app/2VWBcCd Pack 2: go.bsky.app/Bw84Hmc DM if you want to be included (or nominate people who should be!)

1011956

Reposted by Pete Shaw

Kuzman Ganchev @ganchev.bsky.social · 11/11/2024

Wanted to share that Varun Godbole recently released a prompting playbook. The title says prompt tuning, but this is text prompts, not soft prompts. github.com/varungodbole...

github.com

GitHub - varungodbole/prompt-tuning-playbook: A playbook for effectively prompting post-trained LLMs

A playbook for effectively prompting post-trained LLMs - varungodbole/prompt-tuning-playbook

0147

Reposted by Pete Shaw

Maike Osborne @maosbot.bsky.social · 09/11/2024

New here? Interested in AI/ML? Check out these great starter packs! AI: go.bsky.app/SipA7it RL: go.bsky.app/3WPHcHg Women in AI: go.bsky.app/LaGDpqg NLP: go.bsky.app/SngwGeS AI and news: go.bsky.app/5sFqVNS You can also search all starter packs here: blueskydirectory.com/starter-pack...

66557212

Pete Shaw @ptshaw.bsky.social · 11/11/2024

Getting set up on Bluesky today!

050

Reposted by Pete Shaw

Jacob Eisenstein @jacobeisenstein.bsky.social · 24/10/2024

I’m pretty excited about this one! ALTA is A Language for Transformer Analysis. Because ALTA programs can be compiled to transformer weights, it provides constructive proofs of transformer expressivity. It also offers new analytic tools for *learnability*. arxiv.org/abs/2410.18077

arxiv.org

ALTA: Compiler-Based Analysis of Transformers

We propose a new programming language called ALTA and a compiler that can map ALTA programs to Transformer weights. ALTA is inspired by RASP, a language proposed by Weiss et al. (2021), and Tracr (Lin...

25316