iiinova — where your digital self keeps evolving

sardean @sardean.bsky.social · 07/10/2026

I really like the idea of using random search as a bridge!

sardean @sardean.bsky.social · 07/10/2026

I think it worked, especially since the course is targeted at students after a supervised ML course. But the transition from that to the sequential setting felt kind of onerous with all the MDP background up front: sdean.website/cs4789.html

sardean @sardean.bsky.social · 07/10/2026

very cool! In my Intro RL course this year I also avoided MDPs (at first), but with a PG focus instead in a one step setting (bsky.app/profile/sard...)

sardean @sardean.bsky.social · 29/09/2026

a month in, and I finally managed to update the public facing course calendar: sdean.website/cs4789.html

sdean.website

CS 4789/5789: Introduction to Reinforcement Learning

001

sardean @sardean.bsky.social · 15/09/2026

Automatic feedback control is a fun area for this. On the one side, you have pen and paper proofs/understanding/extremely simple models. On the other, you have formally verified systems keeping airplanes in the sky.

sardean @sardean.bsky.social · 10/09/2026

Do you think credit is important primarily for attribution? For me, it's importance is in constructing and maintaining the record about the provenance of ideas. Attribution is only a side effect of this type of knowledge maintenance/construction.

011

sardean @sardean.bsky.social · 31/08/2026

in March, I gave an intro for Cornell's "Science on Screen" program for a screening of 2001: A Space Odyssey. I have been meaning to share what I said, and now seems as good a time as ever.

leaflet.pub

Science on Screen: AI, from "2001: A Space Odyssey" to Today

161

sardean @sardean.bsky.social · 30/08/2026

(previously: bsky.app/profile/sard...)

000

sardean @sardean.bsky.social · 30/08/2026

if you start an RL course with policy gradient, what does it look like to eventually introduce MDPs? here is my attempt: sdean.website/rl-manuscrip...

sdean.website

100

sardean @sardean.bsky.social · 04/08/2026

100% agree. I've been surprised by how quickly I personally pick up on (and hence become annoyed by) little stylistic quirks, even though the homogenization is something I would have predicted in theory

000

sardean @sardean.bsky.social · 04/08/2026

It seems to me like the time it takes to diffuse would also delay the time til a quirk really appears in training data (including that provided by raters)...hard for me to say actually which "side" I would credit with being faster

sardean @sardean.bsky.social · 04/08/2026

why do you think LLMs can update much faster?

sardean @sardean.bsky.social · 19/07/2026

(I've posted before about how the rest of the course will look: bsky.app/profile/sard...)

sardean @sardean.bsky.social · 19/07/2026

imagining how to start an advanced undergraduate reinforcement learning course without MDPs but with policy gradients. Here is what I came up with so far, feedback is welcome! sdean.website/rl-manuscrip...

sdean.website

and here is the revamped RL course outline!

000

the final unit (ch 6-8) is about "exploration": bandits, planning (where we will touch on MPC, LQR, and maybe even Montecarlo tree search), and imitation learning.

100

we start with a dive into policies, estimation, gradients, variance reduction, all in the contextual bandit setting. then, using auto-regressive policies as a bridge, we consider sequential decision-making and MDPs, value iteration, and policy iteration (which is where we revisit policy gradient)

100

working on some lecture notes to guide my revamped undergraduate reinforcement learning course this fall. (as you can from the page numbers, its still mostly blank.) we are going to dive into policy gradients before uttering the name Markov 😲

screenshot of pdf contents:

1 Introduction and Overview 3
2 Policies, Optimization, and Gradients 4
3 Sequential Decisions 18
4 Value Iteration 20
5 Policy Iteration 21
6 Online Decision-making 22
7 Receding Horizon Planning 23
8 Learning from Experts 24

sardean @sardean.bsky.social · 08/07/2026

then, I asked Yahya if he thought he could prove it in time for L4DC submissions (2 weeks). Didn't quite turn out that simple, but we got it in time for ICML a few months later!

000

sardean @sardean.bsky.social · 08/07/2026

our main results show that (under suitable conditions), two layer autoregressive linear models, trained on input-output data, recover Kalman filter state estimates at the hidden layer. back in October, I tried to find this result in the literature (I was sure it must already exist)

100

sardean @sardean.bsky.social · 08/07/2026

tomorrow at ICML my student Sunmook will present a poster on this fun paper, led by postdoc Yahya Sattar: arxiv.org/abs/2606.12691 come for a rigorous view of "representation learning" and the latent space of "world models", stay for the linear systems theory and matrix factorization!

arxiv.org

Two-Layer Linear Auto-Regressive Models Estimate Latent States

Auto-regressive models have emerged as powerful tools for sequential data, from language to video. Understanding how and why these models learn latent representations remains an open theoretical quest...

141

sardean @sardean.bsky.social · 07/07/2026

tomorrow (today over in Korea) Haruka will be presenting this paper in the ICML. working on this project with her (and folks at Meta) is part of what convinced me to revamp my undergraduate RL course (more on that later). check out the poster at the Wednesday poster session! arxiv.org/abs/2605.26385

arxiv.org

Credit-assigned Policy Gradient for Early Stage Retrieval in Two-stage Ranking

Large-scale search, recommendation, and retrieval-augmented generation (RAG) systems typically employ a two-stage architecture: an early-stage ranker (ESR) generates a candidate set, which is subseque...

1113

sardean @sardean.bsky.social · 08/06/2026

It's also a fun in class activity to run an approximation of this live! slides.com/sarahdean-2/...

sardean @sardean.bsky.social · 30/03/2026

www.nytimes.com/2025/12/03/m...

sardean @sardean.bsky.social · 30/03/2026

Wait, and the usage of "tapestry" and "whisper" too!

100

sardean @sardean.bsky.social · 04/03/2026

Naming things at the edge of scientific understanding is hard I guess! I have to relearn every few years that Vitamin K is not the same thing as potassium.

020

sardean @sardean.bsky.social · 04/03/2026

wait, so vitamins A, B, C, and D were just named in their order of discovery?

sardean @sardean.bsky.social · 18/02/2026

on the other hand, learning about new productivity hacks and tools is also, in my opinion, a cope to avoid doing actual work on things that feel hard and scary. signed, someone was frequently targeted by pocket (RIP) recommendations on firefox new tabs

sardean @sardean.bsky.social · 01/02/2026

Yup, android! When I discovered this feature back on my Pixel 4a it became a requirement for me for any future phone

sardean @sardean.bsky.social · 01/02/2026

I actually prefer a phone - extra dim, night mode, amber text for bedtime reading is hard to beat. I would consider buying an e reader if I thought it would compete on this.

sardean @sardean.bsky.social · 22/01/2026

almost a decade! how can that be true when I was almost certainly in high school or middle school a decade a go. (I will not be performing subtraction to verify)

sardean @sardean.bsky.social · 14/01/2026

for some reason, Stephen Wolfram's walking desk strategy has not caught on despite his amazing 2019 blog post writings.stephenwolfram.com/2019/02/seek...

000

sardean @sardean.bsky.social · 09/01/2026

the idea of "deli papers" being an issue is also new to me! The type of "thinly sliced contribution" that I expected to read about is the "minimum publishable unit" approach to scoping papers and projects. But that's distinct from submitting multiple "slices" at once. (Also probably harder to solve)

sardean @sardean.bsky.social · 06/01/2026

the slip from "this thing causes problems for some people" to "everyone is better off avoiding this" is so prevalent in baby safety recommendations. I'm not sure it's always wrong! But it's definitely not always right.

041

sardean @sardean.bsky.social · 05/01/2026

I'm curious why SVMs in particular as the place for introducing kernels to ML students - what about least squares/ridge regression?

100

sardean @sardean.bsky.social · 13/12/2025

Fwiw, I don't think the article refutes a pro-Sacks view! I read it before seeing any discourse and didn't find it to be inflammatory or damning in any way. Just more insight into Oliver Sacks as a person.

130

Reposted by sardean

Aaron Roth @aaroth.bsky.social · 08/11/2025

If you work at the intersection of CS and economics (or think your work is of interest to those who do!) consider submitting to the ESIF Economics and AI+ML meeting this summer at Cornell: www.econometricsociety.org/regional-act...

econometricsociety.org

2026 ESIF Economics and AI+ML Meeting - The Econometric Society

2026 ESIF Economics and AI+ML Meeting (ESIF-AIML2026) June 16-17, 2026 Cornell University Department...

176

sardean @sardean.bsky.social · 24/10/2025

this includes Markov chains, Kalman filtering, optimal linear-quadratic control... can't seem to get away from linear models and quadratic costs

sardean @sardean.bsky.social · 24/10/2025

having just finished the lecture portion of my PhD level ML in Feedback Systems course, and it turns out that everything I understand in ML/control is basically linear least squares

sardean @sardean.bsky.social · 24/08/2025

Given the NSF funding priorities, everything's got to be AI these days. That or quantum I guess.

180

sardean @sardean.bsky.social · 18/08/2025

I launched these balloons as part of an outreach program: bowers.cornell.edu/news-stories... it was a lot of fun! Turns out weather balloons are a great way to talk about both classic and modern "AI"

sardean @sardean.bsky.social · 18/08/2025

And in early August, balloon #2 ended it's journey in the Black Sea 🫡

sardean @sardean.bsky.social · 14/08/2025

small conferences are lots of fun! consider joining us next June in LA, and share with folks who might be interested.

sardean @sardean.bsky.social · 01/08/2025

Actually, I launched two balloons. But the first one crashed into the ocean off the coast of Nova Scotia due to a ballast issue. RIP

100

sardean @sardean.bsky.social · 01/08/2025

two weeks ago, I launched a weather balloon. After a loop de loop over the middle of the Atlantic, it's currently making its way over north Africa. windbornesystems.com/balloon-soci...

A weather balloon's path from North America over the Atlantic to North Africa

sardean @sardean.bsky.social · 21/07/2025

E.g. at netflix "we initially did not observe significant improvements in performance over well-tuned non-deep-learning approaches. Only when we added numerous features of heterogeneous types to the input data, deep-learning models did start to shine in our setting" ojs.aaai.org/aimagazine/i...

ojs.aaai.org

Deep Learning for Recommender Systems: A Netflix Case Study | AI Magazine

sardean @sardean.bsky.social · 21/07/2025

But more generally, this seems like a repeated pattern for deep learning. One of the main advantages is how flexible the models are, and how many disparate types of inputs they can be trained to use. This is in contrast to physics based or simpler ML models.