iiinova — where your digital self keeps evolving

Reposted by Sasha Rush

Naomi Saphra @nsaphra.bsky.social · 05/02/2025

If you're in Berkeley or like a nice streamed talk, I'm about to give a talk at the Simons Institute today: "You Know It Or You Don’t: Compositionality and Phase Transitions in LMs". Tune in at 4PM pacific!

simons.berkeley.edu

Talk By

No abstract available.

1213

Sasha Rush @srushnlp.bsky.social · 04/02/2025

What to know about DeepSeek youtu.be/0eMzc-WnBfQ?... In which we attempt to figure out MoE, o1, scaling, tech reporting, modern semiconductors, microeconomics, and international geopolitics.

youtu.be

How DeepSeek Changes the LLM Story

YouTube video by Sasha Rush 🤗

19714

Sasha Rush @srushnlp.bsky.social · 30/01/2025

For reasons, I find myself thinking a lot about the history of US/USSR Cold War science, particularly in applied math. Does anyone have a recommendation for a good book on this topic?

471

Sasha Rush @srushnlp.bsky.social · 07/01/2025

10 short videos about LLM infrastructure to help you appreciate Pages 12-18 of the DeepSeek-v3 paper (arxiv.org/abs/2412.19437) www.youtube.com/watch?v=76gu...

youtube.com

Flash LLMs: Pipeline Parallel

YouTube video by Sasha Rush 🤗

3285

Sasha Rush @srushnlp.bsky.social · 06/01/2025

I tried reading this book, and I was just shocked at how little insight it had, and it's sheer inability to focus. The fact that it is being recommended to policy makers... www.gatesnotes.com/The-Coming-W...

gatesnotes.com

10391

Sasha Rush @srushnlp.bsky.social · 27/12/2024

I'm going to do a live coding stream for the next couple of hours. We'll start by running through some WebGPU tutorials. Can also talk about some AI stuff. www.youtube.com/watch?v=sqKq...

youtube.com

Python + WebGPU

YouTube video by Sasha Rush 🤗

1214

Reposted by Sasha Rush

Lewis Tunstall @lewtun.bsky.social · 16/12/2024

We outperform Llama 70B with Llama 3B on hard math by scaling test-time compute 🔥 How? By combining step-wise reward models with tree search algorithms :) We're open sourcing the full recipe and sharing a detailed blog post 👇

410921

Sasha Rush @srushnlp.bsky.social · 11/12/2024

As coding LLMs get faster at inference, iterating verification-in-the-loop tests becomes the bottleneck for coding agents. Probably need quite different programming systems for these settings, or even things like "batchable" runtimes, whatever that means.

2100

Reposted by Sasha Rush

antonio vergari ⚔️ short-circuiting @nolovedeeplearning.bsky.social · 06/12/2024

We organised a lively poster fest with many students rehearsing for the upcoming @neuripsconf.bsky.social next week and others discussing their cool works! Thanks to #GAIL, the #Generative #AI lab in #Edinburgh for sponsoring the event!

0447

Reposted by Sasha Rush

Andrew Gordon Wilson @andrewgwils.bsky.social · 05/12/2024

I wanted to make my first post about a project close to my heart. Linear algebra is an underappreciated foundation for machine learning. Our new framework CoLA (Compositional Linear Algebra) exploits algebraic structure arising from modelling assumptions for significant computational savings! 1/4

313821

Reposted by Sasha Rush

Margaret Mitchell @mmitchell.bsky.social · 03/12/2024

Proud of my amazing colleague @sashamtl.bsky.social for her much deserved recognition on advancing the science of AI energy use. BBC100: www.bbc.co.uk/news/resourc... Fast Company: www.fastcompany.com/91233692/why... Sasha has been working tirelessly moving things fwd--endurance & brilliance in one.

Screenshot of BBC 100 picture of Sasha and blurb; linked in post.

2335

Reposted by Sasha Rush

Kempner Institute at Harvard University @kempnerinstitute.bsky.social · 03/12/2024

NEW: we have an exciting opportunity for a tenure-track professor at the #KempnerInstitute and the John A. Paulson School of Engineering and Applied Sciences (SEAS). Read the full description & apply today: academicpositions.harvard.edu/postings/14362 #ML #AI

02019

Reposted by Sasha Rush

Nathan Lambert @natolambert.bsky.social · 03/12/2024

We're hiring another predoctoral researcher for my team at Ai2/OLMo next year. The goal of this position is to mentor and grow future academic stars of NLP/AI over 1-2 years before grad school. This ends up being people done with BS or MS who want to continue to a PhD soon. buff.ly/49nuggo

6547

Sasha Rush @srushnlp.bsky.social · 02/12/2024

Is there a community that writes RL-first programming languages? Something like (Num)Pyro that takes seriously the idea of separating the policy specification from the learning process.

2160

Reposted by Sasha Rush

Todd Morrill @toddmorrill.bsky.social · 01/12/2024

ICYMI, @srushnlp.bsky.social recently gave a nice talk speculating about the methods/data used to train OpenAI's o1 model. The key idea seems to be scaling up chain-of-thought (CoT) generation using auxiliary verifier models that can give feedback on the correctness of the generation. 1/N

1101

Reposted by Sasha Rush

François Fleuret @francois.fleuret.org · 26/11/2024

My deep learning course at the University of Geneva is available on-line. 1000+ slides, ~20h of screen-casts. Full of examples in PyTorch. fleuret.org/dlc/ And my "Little Book of Deep Learning" is available as a phone-formatted pdf (nearing 700k downloads!) fleuret.org/lbdl/

461253247

Reposted by Sasha Rush

William Merrill @lambdaviking.bsky.social · 26/11/2024

A starter pack for research on Formal Languages and Neural Networks! go.bsky.app/eKG7Ua

3414

Reposted by Sasha Rush

International Conference on 3D Vision @3dvconf.bsky.social · 26/11/2024

Hello Bluesky! 🔵 We start our account by having our third guess for Ask Me Anything session #3DV2025AMA! Noah Snavely @snavely.bsky.social from Cornell & Google DeepMind! 🌟 🕒 You have now 24 HOURS to ask him anything — drop your questions in the comments below! Keep it engaging but respectful!

134514

Sasha Rush @srushnlp.bsky.social · 25/11/2024

Rare personal tweet: Subletting our furnished apartment in Brooklyn for the spring at a significant discount. It's quite nice and in a fun location. under price. Email me know if you are interested, I will send pictures.

0257

Sasha Rush @srushnlp.bsky.social · 21/11/2024

Several incredible NeurIPS tutorials this year. Worth navigating through the Swifties.

1406

Reposted by Sasha Rush

Edoardo Ponti @edoardo-ponti.bsky.social · 20/11/2024

Another nano gem from my amazing student Piotr Nawrot! A repo & notebook on sparse attention for efficient LLM inference: github.com/PiotrNawrot/... This will also feature in my #NeurIPS 2024 tutorial "Dynamic Sparsity in ML" with André Martins: dynamic-sparsity.github.io Stay tuned!

A sparse mask of attention scores based on VerticalAndSlashAttention and a plot of loss vs sparsity ratio for various methods.

2428

Reposted by Sasha Rush

Nathan Lambert @natolambert.bsky.social · 21/11/2024

I've spent the last two years scouring all available resources on RLHF specifically and post training broadly. Today, with the help of a totally cracked team, we bring you the fruits of that labor — Tülu 3, an entirely open frontier model post training recipe. We beat Llama 3.1 Instruct. Thread.

821343

Sasha Rush @srushnlp.bsky.social · 21/11/2024

Discrete diffusion has become a very hot topic again this year. Dozens of interesting ICLR submissions and some exciting attempts at scaling. Here's a bibliography on the topic from the Kuleshov group (my open office neighbors). github.com/kuleshov-gro...

github.com

GitHub - kuleshov-group/awesome-discrete-diffusion-models: A curated list for awesome discrete diffusion models resources.

A curated list for awesome discrete diffusion models resources. - kuleshov-group/awesome-discrete-diffusion-models

17610

Sasha Rush @srushnlp.bsky.social · 21/11/2024

A disproportionate number of sucessful CS academics have some intense cardio hobby. Took me some years to understand.

6933

Reposted by Sasha Rush

Ramon Astudillo @ramon-astudillo.bsky.social · 19/11/2024

I did a starter pack of people in New York (City) working on ML/AI. Please distribute and feel free to self nominate! go.bsky.app/BoEtagz

428919

Reposted by Sasha Rush

Naomi Saphra @nsaphra.bsky.social · 02/07/2023

hard for me to get engagement on this platform but it’s ok I’m just waiting for all the AI researchers to show up so I can go mega viral saying shit like “who called it Xavier Initialization and not Weighting For Glorot”

5777

Sasha Rush @srushnlp.bsky.social · 18/11/2024

I keep reading interviews about how impressively smart and accomplished the AlphaProof team is. But I would be way more impressed if they were just like "we're super bad at math, who cares".

1260

Sasha Rush @srushnlp.bsky.social · 15/11/2024

Guest Video: Should I do a Postdoc? - Niloofar Mireshghallah It's the time of the year when a lot of CS PhDs are thinking about next steps in the process. Niloofar gives a thoughtful, personal view on postdoctoral life. www.youtube.com/watch?v=O7ps...

youtube.com

Should I do a Postdoc? - Niloofar Mireshghallah

YouTube video by Sasha Rush 🤗

071

Sasha Rush @srushnlp.bsky.social · 12/11/2024

Talk: Speculations on Test-Time Scaling A tutorial on the technical aspects behind OpenAI's o1 and open research questions in this space. youtu.be/6PEJ96k1kiw Slides+bibliography: github.com/srush/awesom...

youtu.be

Speculations on Test-Time Scaling (o1)

YouTube video by Sasha Rush 🤗

18826

Sasha Rush @srushnlp.bsky.social · 11/11/2024

I'll be at EMNLP this week. I would love to chat. I'm interested in learning for reasoning, alternative architectures, embedding models, text diffusion, and where NLP is going as a field. calendly.com/srush-resear...

calendly.com

Sasha EMNLP - Sasha Rush

070

Reposted by Sasha Rush

Adam L @adam-lg.bsky.social · 02/04/2024

"The Elements of Differentiable Programming" link: arxiv.org/abs/2403.14606 Basically: "autodiff - it's everywhere! what is it, and how do you use it?" seems like a good resource for anyone interested in data science, machine learning, "ai," neural nets, etc #blueskai #stats #mlsky

Artificial intelligence has recently experienced remarkable advances, fueled by large models, vast datasets, accelerated hardware, and, last but not least, the transformative power of differentiable programming. This new programming paradigm enables end-to-end differentiation of complex computer programs (including those with control flows and data structures), making gradient-based optimization of program parameters possible. As an emerging paradigm, differentiable programming builds upon several areas of computer science and applied mathematics, including automatic differentiation, graphical models, optimization and statistics. This book presents a comprehensive review of the fundamental concepts useful for differentiable programming. We adopt two main perspectives, that of optimization and that of probability, with clear analogies between the two.

03213

Sasha Rush @srushnlp.bsky.social · 08/11/2024

Heard we are moving NLP twitter over here now. Seems like a good time, I'll try to cross post.

51006

Reposted by Sasha Rush

Yoav Artzi @yoavartzi.com · 16/10/2023

Introducing COLM (colmweb.org) the Conference on Language Modeling. A new research venue dedicated to the theory, practice, and applications of language models. Submissions: March 15 (it's pronounced "collum" )

22613

Sasha Rush @srushnlp.bsky.social · 04/10/2023

Autodiff Puzzle (v0.2, github.com/srush/Autodi...) - How come my Calculus teacher never taught me the derivative of sort()? 20 one-line puzzles for differentiating all the things.

030