iiinova — where your digital self keeps evolving

nadia @nadia9.bsky.social · 28/09/2026

brb I’ll just go write the book I’ve always wanted to write on this :) In the meantime, this LLM viz I built could be fun. It’s fairly technical but clicking through it at least gives a feel for “lots of nested boxes of math which eventually spit out a next token” nadiawadzinski.com/annotated-llm/

nadiawadzinski.com

The annotated LLM

Explore an LLM (Qwen3) under the hood, with a zoomable visualization alongside PyTorch code.

030 0

nadia @nadia9.bsky.social · 27/09/2026

Hindsight makes it so hard to evaluate how reasonable past predictions were given the information at the time (especially if they were correct!). But I’d point out that in fact just scaling pretraining was not enough; it required training innovations like huge amounts of RLVR to get us to today.

011 0

nadia @nadia9.bsky.social · 27/09/2026

<history textbook intonation> “By fall 2026, LLMs had progressed to solving millennium prize math problems, and to first forays into kitchen navigation. Famously, 2027 showed that successfully baking a three-layer cake in a messy kitchen was more difficult than resolving the Riemann hypothesis.”

2130 0

nadia @nadia9.bsky.social · 27/09/2026

and this isn’t to discount all the cool innovations in architecture since then, like RoPE, MoE, sparse attention and sliding window and all the other new attention variants. But they’re variations and optimizations within the same overall architecture (phylum?) of embeddings + stacked transformers.

010 0

nadia @nadia9.bsky.social · 27/09/2026

Oh sure, but I think that was kind of a delightfully outrageous belief to hold at the time (and yet… it was correct!). And of course there *were* major innovations required to get here, it’s not just scaling. But they were in training and not architecture: RLVR (+ train of thought) is the big one.

110 0

nadia @nadia9.bsky.social · 27/09/2026

It’s hard to overstate this. Even if you believed in 2019 that by 2026 AI would be solving millennium prize math problems (or navigating kitchens) you would surely (surely!) have expected it to be using an utterly different architecture than GPT-2. Yet here we are, still stacking transformer layers.

5672 0

nadia @nadia9.bsky.social · 15/11/2023

oh yes that’s of course the real reason I’m wearing it - I love my Barbour jacket despite its Tory lady vibes.

110 0

nadia @nadia9.bsky.social · 11/08/2023

thing is, I’m not a lesbian, have never been a lesbian, and yet, in practice these days, I’m only into other trans girls. So maybe he needs a whole book on T4T sociology…

120 0

nadia @nadia9.bsky.social · 10/07/2023

it’s pleasant how even mentioning the mommy feed makes you end up on the mommy feed

010 0