David Bau @davidbau.bsky.social · 06/10/2026On one hand, failure to introspect is unsurprising. You train it to say A/B; why would you expect it to expound on thoughts? On the other hand many others have long noticed that frontier AI *CAN* introspect. Maybe Qwen is just too dumb to do it... 🧵 → bsky.app/profile/kma... 120
David Bau @davidbau.bsky.social · 06/10/2026Next: ask it to introspect. What does Gregor like in washing machines? It will happily say "I think X". But it is TERRIBLE at it! Qwen acts as a stochastic parrot, spewing words about its thoughts that are NONSENSE. Unrelated to what it actually learned in A/B training. 🧵→ 140
David Bau @davidbau.bsky.social · 06/10/2026After just a bit of training Qwen totally gets it. Soon enough it can correctly answer A or B on new "Gregor Samsa washing machine puzzles". It generalizes. Unsurprising. Machine learning works. but ...🧵 → 120
David Bau @davidbau.bsky.social · 06/10/2026The craziest experiment in my lab is a new setup by @diatkinson.bsky.social where he has found a way induce introspection on open LMs using Dillon Plunkett's protocol, that lets him look inside introspection. It's nuts. Here's how the experiment works... 🧵 → 1121
David Bau @davidbau.bsky.social · 18/07/2026Is it possible to write 100,000 lines of code well, if you do not read it? Let's go Hunting Zombies! davidbau.com/archives/20... In this post I dive into the code of two AI agent contestants in the Teleport coding challenge to learn their secrets. Very fun. And also instructive. 270
David Bau @davidbau.bsky.social · 17/06/2026Mousing masks a few "gaze" attention heads to a spotlight and instantly shows the causal effects. Guide the model to the striped canopies, and then shift its attention to the autumn leaves, the fields of grain... Then check out the preprint! gaze.baulab.info 000
David Bau @davidbau.bsky.social · 17/06/2026You can and read the preprint at gaze.baulab.info/#demo But first on that page scroll to the demo with WebGPU machine (most modern laptops+browsers) and click on "start demo". It's worth the wait as it downloads the 2B VLM. 100
David Bau @davidbau.bsky.social · 17/06/2026Oh man! I love this preprint and also the website Rohit made to demo it. The gaze of a VLM is mediated by a much smaller set of attention heads than the full set, as if "conscious" attention is a small subset of "all attention heads". His demo lets you steer these in realtime. 180
David Bau @davidbau.bsky.social · 05/06/2026📅 Key dates (Summer 2026): Applications close June 21 Competition begins June 29 Submissions due July 26 Awards event to invitees, Boston, Aug 25 Register: aletheias-quest.github.io Discord: discord.gg/4NX3sUC4ge 020
David Bau @davidbau.bsky.social · 05/06/2026"You're right to call me on that!" Can you catch an AI in the act of lying? Register below to enter our AI lie-detection contest. AI lies are a big problem. The frontier labs have all worked hard to fight AI deception. They all try to monitor their AIs for it. 173
David Bau @davidbau.bsky.social · 06/05/2026The leaderboard is open. Click "play" to play one of the submitted NetHack ports, and "Tests" to see what points are being missed. mazesofmenace.ai/ 100
David Bau @davidbau.bsky.social · 06/05/2026In the contest, you can try your hand at steering the evolution of large-scale LLM "religion". I have created a porting sandbox, a skeleton start, a leaderboard, and visualization tools. I still do not know if agents can do it. Can you port NetHack 5? 100
David Bau @davidbau.bsky.social · 06/05/2026The agents invented an idea, "sparse boundary frames" and defended it with rigorous authority. No such thing: They had created their own religion. With relentless efficiency, the agents infected 200K lines of code with their wrong ideas. Eventually I had to throw it all away. 100
David Bau @davidbau.bsky.social · 06/05/2026LLMs are great at porting. Porting Hack and Rogue was easy: eighty-five minutes for Rogue. But then I pointed the agents at NetHack and watched them get stuck for three weeks at 18 failing tests. They were doing a hundred commits a day. But they couldn't move the bottom line. 100
David Bau @davidbau.bsky.social · 06/05/2026The Teleport Contest is open. Port NetHack 5.0 from C to JavaScript, bit-exactly. Same screen, every keystroke. Any approach: LLM agents, hand-coded, transpiler, hybrid. Live leaderboard, two phases through December. mazesofmenace.ai/announcement 260
David Bau @davidbau.bsky.social · 03/05/2026You score points by producing a readable, maintainable JS version whose input-output behavior matches official NetHack 5.0 turn-by-turn on a held-out set of gameplay sequences. We will define a starter template and some public test sessions. 110
David Bau @davidbau.bsky.social · 03/05/2026I am very curious if frontier orchestration systems like steve-yegge.bsky.social's Gastown or @ryancarson.com's Antfarm can crack it quickly. If there is interest, Alex Boruch-Gruszecki and I will flesh out contest rules this week and run a leaderboard. 110
David Bau @davidbau.bsky.social · 03/05/2026The question: Can a swarm of LLM coding assistants enable a single person to work with a program of this scale and complexity? Can Claude, Codex, and Gemini understand the mammoth codebase well enough to convert it all to JS? Sure? How quickly? Anybody want to race? 110
David Bau @davidbau.bsky.social · 03/05/2026Under active development over four decades and tracing its origins to Rogue (1980) and Hack (1982), the 442,901 lines of C and Lua in NetHack 5.0 have become a deeply interconnected system with many layers of intricate behavior. 110
David Bau @davidbau.bsky.social · 03/05/2026NetHack is one of the most complex and longest-lived open source programs ever written, and after 46 years, v5.0 shipped today. www.nethack.org/common/inde... And ... it is a VERY cool large codebase to work with in the LLM era. 1103
David Bau @davidbau.bsky.social · 23/03/2026The gap between Hack and NetHack taught me something about the future of CS. Worth thinking about. Does Computer Science Still Exist? davidbau.com/archives/20... 151
David Bau @davidbau.bsky.social · 23/03/2026NetHack, the grandchild of Hack, is here: mazesofmenace.net/ I didn't write the code. Claude and Codex did. Rogue took 85 minutes. Hack took eight hours. NetHack (420,000 lines of C) has been grinding for two months and STILL isn't done! 110
David Bau @davidbau.bsky.social · 23/03/2026Here is a 1982-rendition of the Logo programming language: mazesofmenace.net/logo/ Why? Because Logo was the community that connected all of them. 120
David Bau @davidbau.bsky.social · 23/03/2026Rogue (1980) was the original that "Roguelikes" are like, by UCSC students Michael Toy and Glenn Wichman. Play it here: mazesofmenace.net/rogue/ Each one comes with a history of the people who made it. 110
David Bau @davidbau.bsky.social · 23/03/2026I ported these games to JavaScript so you can play them. Try Hack (1982) here: mazesofmenace.net/hack/ This is the original gameplay by high-schooler Jay Fenlason. 120
David Bau @davidbau.bsky.social · 23/03/2026In 1982, high school students in Sudbury, Mass. wrote a dungeon game called Hack. They had Atari 800s and Logo and an obsession with a Unix game called Rogue that most of them had never seen. I grew up one town over with the same computers and the same obsession. 171
David Bau @davidbau.bsky.social · 23/02/2026@natalieshapira.bsky.social and team have written up enlightening case studies here. It's all cross-referenced with detailed activity logs. Well worth a read: agentsofchaos.baulab.info/report.html www.researchgate.net/publication... 031
David Bau @davidbau.bsky.social · 23/02/2026Are we all Agents of Chaos in AI? (Hope not!) In recent weeks using OpenClaw has taught us a lot about this wooly new kind of autonomous software agent. Its valuable to see what @NatalieShapira, @wendlerch et al. have seen: agentsofchaos.baulab.info/ 2167
David Bau @davidbau.bsky.social · 21/02/2026Preprint, code, and model weights at hapax.baulab.info 120
David Bau @davidbau.bsky.social · 21/02/2026Without induction, can the LM think? Well, it can't copy text verbatim very well: no surprise. But, huge surprise! It becomes good at things like translating from Spanish to English, learning some things like this FASTER WITHOUT induction. Read more in Kerem's X thread x.com/keremsahin2... 140
David Bau @davidbau.bsky.social · 21/02/2026And if you knock out induction heads by blocking their attention patterns, induction still emerge, shifting their attention aside to avoid your masks. They beat you at whack-a-mole. 40% of natural text copies neaby ngrams + LMs really want to exploit this. 140
David Bau @davidbau.bsky.social · 21/02/2026Kerem's induction-removal was going to be the "first step" of a bigger study of LM mechanisms. But he soon discovered: it is not so easy to knock out induction. Whenever you try, a bit of fine-tuning brings the heads roaring back. More each time, over and over: 130
David Bau @davidbau.bsky.social · 21/02/2026How do you knock the induction heads out of an LM while preserving its ability to think? Is it even possible? @keremsahin22.bsky.social's work is worth reading if you haven't seen it yet. hapax.baulab.info 1276
David Bau @davidbau.bsky.social · 28/01/2026If Sam Altman can't listen to his moral convictions, he will listen to his employees. It is important for tech employees to make it clear that we will not accept making AI for authoritarianism. davidbau.github.io/poetsandnurses On GitHub. PRs welcome. github.com/davidbau/poetsandnurses 010
David Bau @davidbau.bsky.social · 27/01/2026I left Google in 2015 to pursue the insight that "Volo Ergo Sum." That the central challenge in AI is how to amplify human agency. This is not easy. Do you think AI will ever be superhuman at taking responsibility for what should be? Read more: davidbau.com/archives/20... 100
David Bau @davidbau.bsky.social · 27/01/2026On the other hand we have @dhadfieldmenell.bsky.social who draws a line at normative judgments. x.com/dhadfieldme... 100
David Bau @davidbau.bsky.social · 27/01/2026That question is in the air. Today @polynoamial.bsky.social pokes fun at the series of "AI can't do what a human brain can do" predictions. Of course AI can do anything a human brain can do, Noam argues. Including making wise decisions. x.com/polynoamial... 100
David Bau @davidbau.bsky.social · 27/01/2026The Art of Wanting. About the question I see as central in AI ethics, interpretability, and safety. Can an AI take responsibility? I do not think so, but *not* because it's not smart enough. davidbau.com/archives/20... 193
David Bau @davidbau.bsky.social · 26/01/2026What should academics be doing right now? I have been writing up some thoughts on what the research says about effective action, and what universities specifically can do. davidbau.github.io/poetsandnurs... It's on GitHub. Suggestions and pull requests welcome. github.com/davidbau/poe... 03715
David Bau @davidbau.bsky.social · 25/01/2026From induction to FVs, every ICL mechanism we've pinned down is fuzzy copying. Is copying all there is? @ericwtodd.bsky.social trained on groups where tokens have no fixed meaning and found a basket of mechanisms beyond copying. Watch them emerge, a grokking cascade! ↓ bsky.app/profile/eri... 060
David Bau @davidbau.bsky.social · 12/01/2026I can't read Chinese, but my family has old genealogy documents I've always wanted to understand. Claude and Gemini helped me build an interactive reader to explore the calligraphy character by character. I can finally read my great-grandfather's epitaph. Try it: davidbau.com/archives/202... 0254
David Bau @davidbau.bsky.social · 06/01/2026My vibe-coded Mandelbrot viewer is 40x faster now! New GPU synchronization tricks go outside the design intent of WebGPU specs. But the real story: Claude tells me what happens in the AGI break room. What superhuman AGIs say when the boss is not around: davidbau.com/archives/202... 030
David Bau @davidbau.bsky.social · 18/12/2025I have been teaching myself to vibe code. Watch Claude Code grow my 780 lines to 13,600 - mandelbrot.page/coverage/ca... Two fundamental rules for staying in control: davidbau.com/archives/20... 2215
David Bau @davidbau.bsky.social · 11/12/2025At the #Neurips2025 mechanistic interpretability workshop I gave a brief talk about Venetian glassmaking, since I think we face a similar moment in AI research today. Here is a blog post summarizing the talk: davidbau.com/archives/202... 2195
David Bau @davidbau.bsky.social · 06/11/2025When you read the paper, be sure to check out the appendix where @arnab_api discusses how pointer and value data are entangled in filters. And possible applications of the filter mechanism, like as a zero-shot "lie detector" that can flag incorrect statements in ordinary text. 120
David Bau @davidbau.bsky.social · 06/11/2025The neural representations for LLM filter heads are language independent! If we pick up the representation for a question in French, it will accurately match items expressed in the Thai language. 120
David Bau @davidbau.bsky.social · 11/10/2025How embarrassing for me and confusing to the LLM! OK, here it is fixed. Nice thing about workbench is that it just takes a second to edit the prompt, and you can see how the LLM responds, now deciding very early it should be ':' 030
David Bau @davidbau.bsky.social · 11/10/2025The lens reveals: the model does NOT go directly from amore to "amor" or "amour" by just dropping or adding letters! Instead it first "thinks" about the (English) word "love". In other words: LLMs translate using *concepts*, not tokens. 3335
David Bau @davidbau.bsky.social · 11/10/2025Enter a translation prompt: "Italiano: amore, Español: amor, François:". The workbench doesn't just show you the model's output. It shows the grid of internal states that lead to the output. Researchers call this visualization the "logit lens". 1100
