A common take I’ve been hearing is: “LLMs are especially good at math because math is easy to verify”. But that story doesn’t make much sense.…

Q1: WHAT ARE YOU SAYING? A: My claim here is that if you build artificial general intelligence (AGI) via any algorithm that’s choosing actions via RL and/or model-based search and planning—a giant chu...

LLMs get their impressive capabilities from a combination of: (1) Imitative learning from pretraining and SFT data (see my earlier discussion of “LLM pretraining magically transmutes observations into...

My belief is that keeping AI under human control would be an unprecedented global challenge, if it’s even possible at all. Many other people believe that AIs will naturally remain as tools under human...

In this post, intended for a broad audience, I will paint a brief picture of what I’m talking about when I talk about “AGI”. It will seem obvious to many people, and obviously wrong to many others! So...

As my regular readers know (see Intro to Brain-Like-AGI Safety), I’m working on the technical alignment problem for a hypothetical future “brain-like AGI”, with a particular focus on how human social ...

On one side of this debate is Yudkowsky & Soares, who think that (if AI progress continues) we’re on a direct path to egregiously-misaligned, schemin…

ToC: Part 1: In which I use my optical physics background to share some hopefully-uncontroversial observations. Part 2: In which I boldly defy Public Health Orthodoxy on the whole UV situation

Jeremy Howard was recently[1] interviewed on the Machine Learning Street Talk podcast: YouTube link, interactive transcript, PDF transcript. …

Some people say “the brain is a computer”. Other people say “well, the brain is not really a computer, because, like, what’s the hardware versus the software?” I agree: “the brain is a computer” is ki...

This is a rather ambitious series of blog posts, in that I’ll attempt to explain what’s the deal with consciousness, free will, hypnotism, enlightenment, hallucinations, flow states, dissociation, akr...

…Approval Reward is a brain signal that leads to: • Pleasure (positive reward) when my friends and idols seem to have positive feelings about me, or about something related to me, or about what I’m do...

The race to superhuman AI risks extinction, but it's not too late to change course.

“Optical rectennas” (or sometimes “nantennas”) are a technology that is sometimes advertised as a path towards converting solar energy to electricity with higher efficiency than normal solar cells...

There’s a funny thing where economics education paradoxically makes people DUMBER at thinking about future AI. Econ textbooks teach concepts & frames that are great for most things, but counterproduct...

Let’s say I want to type the character μ. I look up the shortcut on the cheat sheet, and see that it’s “[compose key] * m”. So if the compose key is Right-Alt (for example), I would press and release Right-Alt, then *, then m. And μ appears!

Dr. Steven Byrnes, UC Berkeley physics PhD and Harvard physics postdoc, is an AI safety researcher at the Astera Institute, focused on solving the technical AI alignment problem.

I will argue that a large class of reward functions, which I call “behaviorist”, and which includes almost every reward function in the RL and LLM literature, are all doomed to eventually lead to AI t...