iiinova — where your digital self keeps evolving

Sasha Rush @srushnlp.bsky.social · 30/01/2025

19714

These are great recommendations thank you.

030

Sasha Rush @srushnlp.bsky.social · 30/01/2025

For reasons, I find myself thinking a lot about the history of US/USSR Cold War science, particularly in applied math. Does anyone have a recommendation for a good book on this topic?

471

Yeah vertical is kind of dumb, but I thought I try it out.

000

😆, I noticed I have trouble saying that word. however if you listen to your own videos then you will never manage to release anything.

010

Flash LLMs: Pipeline Parallel

10 short videos about LLM infrastructure to help you appreciate Pages 12-18 of the DeepSeek-v3 paper (arxiv.org/abs/2412.19437) www.youtube.com/watch?v=76gu...

youtube.com

3285

Thought this "Bill Gates" guy was on the level.

030

I'll try it out. Good to check once a year to see if I'm secretly an existential risk guy.

030

I mean Microsoft under the table bought his company, right? Luckily there is no indication from the review he read the book.

020

Maybe I'll just use bluesky to rant about topics I'm too scared to talk about on twitter.

2261

The casual conflation of AI with gene editing is intellectual malpractice. These two things have nothing to do with each other!

3140

Would love to read a good book about this topic if anyone wants to give it a try.

150

I've been listening to too much If Books Could Kill, so now I'm convinced these airport books are actually the only thing that matters.

1140

Sasha Rush @srushnlp.bsky.social · 27/12/2024

I tried reading this book, and I was just shocked at how little insight it had, and it's sheer inability to focus. The fact that it is being recommended to policy makers... www.gatesnotes.com/The-Coming-W...

gatesnotes.com

10391

I'm going to do a live coding stream for the next couple of hours. We'll start by running through some WebGPU tutorials. Can also talk about some AI stuff. www.youtube.com/watch?v=sqKq...

youtube.com

Python + WebGPU

Lewis Tunstall @lewtun.bsky.social · 16/12/2024

1214

Reposted by Sasha Rush

We outperform Llama 70B with Llama 3B on hard math by scaling test-time compute 🔥 How? By combining step-wise reward models with tree search algorithms :) We're open sourcing the full recipe and sharing a detailed blog post 👇

410921

Sasha Rush @srushnlp.bsky.social · 12/12/2024

huh, so maybe OCaml should be the target for verifiable generation? I heard you guys have ways to build fast

220

Sasha Rush @srushnlp.bsky.social · 11/12/2024

As coding LLMs get faster at inference, iterating verification-in-the-loop tests becomes the bottleneck for coding agents. Probably need quite different programming systems for these settings, or even things like "batchable" runtimes, whatever that means.

2100

Reposted by Sasha Rush

antonio vergari ⚔️ short-circuiting @nolovedeeplearning.bsky.social · 06/12/2024

We organised a lively poster fest with many students rehearsing for the upcoming @neuripsconf.bsky.social next week and others discussing their cool works! Thanks to #GAIL, the #Generative #AI lab in #Edinburgh for sponsoring the event!

0447

Reposted by Sasha Rush

Andrew Gordon Wilson @andrewgwils.bsky.social · 05/12/2024

I wanted to make my first post about a project close to my heart. Linear algebra is an underappreciated foundation for machine learning. Our new framework CoLA (Compositional Linear Algebra) exploits algebraic structure arising from modelling assumptions for significant computational savings! 1/4

313821

Reposted by Sasha Rush

Margaret Mitchell @mmitchell.bsky.social · 03/12/2024

Proud of my amazing colleague @sashamtl.bsky.social for her much deserved recognition on advancing the science of AI energy use. BBC100: www.bbc.co.uk/news/resourc... Fast Company: www.fastcompany.com/91233692/why... Sasha has been working tirelessly moving things fwd--endurance & brilliance in one.

Screenshot of BBC 100 picture of Sasha and blurb; linked in post.

2335

Reposted by Sasha Rush

Kempner Institute at Harvard University @kempnerinstitute.bsky.social · 03/12/2024

NEW: we have an exciting opportunity for a tenure-track professor at the #KempnerInstitute and the John A. Paulson School of Engineering and Applied Sciences (SEAS). Read the full description & apply today: academicpositions.harvard.edu/postings/14362 #ML #AI

02019

Reposted by Sasha Rush

Nathan Lambert @natolambert.bsky.social · 03/12/2024

We're hiring another predoctoral researcher for my team at Ai2/OLMo next year. The goal of this position is to mentor and grow future academic stars of NLP/AI over 1-2 years before grad school. This ends up being people done with BS or MS who want to continue to a PhD soon. buff.ly/49nuggo

6547

Sasha Rush @srushnlp.bsky.social · 02/12/2024

Answers from Twitter x.com/srush_nlp/st...

x.com

150

Sasha Rush @srushnlp.bsky.social · 02/12/2024

Unfortunately Yoav's question is a bit more interesting and subtle than this talk.

340

Sasha Rush @srushnlp.bsky.social · 02/12/2024

Is there a community that writes RL-first programming languages? Something like (Num)Pyro that takes seriously the idea of separating the policy specification from the learning process.

2160

Reposted by Sasha Rush

Todd Morrill @toddmorrill.bsky.social · 01/12/2024

ICYMI, @srushnlp.bsky.social recently gave a nice talk speculating about the methods/data used to train OpenAI's o1 model. The key idea seems to be scaling up chain-of-thought (CoT) generation using auxiliary verifier models that can give feedback on the correctness of the generation. 1/N

1101

Reposted by Sasha Rush

François Fleuret @francois.fleuret.org · 26/11/2024

My deep learning course at the University of Geneva is available on-line. 1000+ slides, ~20h of screen-casts. Full of examples in PyTorch. fleuret.org/dlc/ And my "Little Book of Deep Learning" is available as a phone-formatted pdf (nearing 700k downloads!) fleuret.org/lbdl/

461252247

Reposted by Sasha Rush

William Merrill @lambdaviking.bsky.social · 26/11/2024

A starter pack for research on Formal Languages and Neural Networks! go.bsky.app/eKG7Ua

3414

Reposted by Sasha Rush

International Conference on 3D Vision @3dvconf.bsky.social · 26/11/2024

Hello Bluesky! 🔵 We start our account by having our third guess for Ask Me Anything session #3DV2025AMA! Noah Snavely @snavely.bsky.social from Cornell & Google DeepMind! 🌟 🕒 You have now 24 HOURS to ask him anything — drop your questions in the comments below! Keep it engaging but respectful!

134514

Sasha Rush @srushnlp.bsky.social · 25/11/2024

Rare personal tweet: Subletting our furnished apartment in Brooklyn for the spring at a significant discount. It's quite nice and in a fun location. under price. Email me know if you are interested, I will send pictures.

0257

Sasha Rush @srushnlp.bsky.social · 25/11/2024

geoffrey pullum has a wonderful rant about his issues strunk and white. I still kind of like it. statmodeling.stat.columbia.edu/2019/02/20/g... As a non-RL person I wish someone would write a new Sutton&Barto the takes into account non-trivial models...

statmodeling.stat.columbia.edu

Geoff Pullum, the linguist who hates Strunk and White, is speaking at Columbia this Friday afternoon | Statistical Modeling, Causal Inference, and Social Science

170

Sasha Rush @srushnlp.bsky.social · 22/11/2024

I know this one is important, but I always found it very hard.

110

Edoardo Ponti @edoardo-ponti.bsky.social · 20/11/2024

Several incredible NeurIPS tutorials this year. Worth navigating through the Swifties.

1406

Reposted by Sasha Rush

Another nano gem from my amazing student Piotr Nawrot! A repo & notebook on sparse attention for efficient LLM inference: github.com/PiotrNawrot/... This will also feature in my #NeurIPS 2024 tutorial "Dynamic Sparsity in ML" with André Martins: dynamic-sparsity.github.io Stay tuned!

A sparse mask of attention scores based on VerticalAndSlashAttention and a plot of loss vs sparsity ratio for various methods.

2428

Reposted by Sasha Rush

Nathan Lambert @natolambert.bsky.social · 21/11/2024

I've spent the last two years scouring all available resources on RLHF specifically and post training broadly. Today, with the help of a totally cracked team, we bring you the fruits of that labor — Tülu 3, an entirely open frontier model post training recipe. We beat Llama 3.1 Instruct. Thread.

821343

GitHub - kuleshov-group/awesome-discrete-diffusion-models: A curated list for awesome discrete diffusion models resources.

Discrete diffusion has become a very hot topic again this year. Dozens of interesting ICLR submissions and some exciting attempts at scaling. Here's a bibliography on the topic from the Kuleshov group (my open office neighbors). github.com/kuleshov-gro...

github.com

A curated list for awesome discrete diffusion models resources. - kuleshov-group/awesome-discrete-diffusion-models

17610

🤣 same techniques work in escaping from bears or terrible NSF reviews

010

Ramon Astudillo @ramon-astudillo.bsky.social · 19/11/2024

A disproportionate number of sucessful CS academics have some intense cardio hobby. Took me some years to understand.

6933

Reposted by Sasha Rush

I did a starter pack of people in New York (City) working on ML/AI. Please distribute and feel free to self nominate! go.bsky.app/BoEtagz

428919

Reposted by Sasha Rush

Naomi Saphra @nsaphra.bsky.social · 02/07/2023

hard for me to get engagement on this platform but it’s ok I’m just waiting for all the AI researchers to show up so I can go mega viral saying shit like “who called it Xavier Initialization and not Weighting For Glorot”

5777

Sasha Rush @srushnlp.bsky.social · 18/11/2024

I keep reading interviews about how impressively smart and accomplished the AlphaProof team is. But I would be way more impressed if they were just like "we're super bad at math, who cares".

1260

Sasha Rush @srushnlp.bsky.social · 15/11/2024

Guest Video: Should I do a Postdoc? - Niloofar Mireshghallah It's the time of the year when a lot of CS PhDs are thinking about next steps in the process. Niloofar gives a thoughtful, personal view on postdoctoral life. www.youtube.com/watch?v=O7ps...

youtube.com

Should I do a Postdoc? - Niloofar Mireshghallah

Sasha Rush @srushnlp.bsky.social · 12/11/2024

071

Talk: Speculations on Test-Time Scaling A tutorial on the technical aspects behind OpenAI's o1 and open research questions in this space. youtu.be/6PEJ96k1kiw Slides+bibliography: github.com/srush/awesom...

youtu.be

Speculations on Test-Time Scaling (o1)