Sign in

NoetherianThing

@noetherianthing.bsky.social
51 followers 70 following 146 posts
PostsRepliesMedia
NoetherianThing @noetherianthing.bsky.social · 11h
1 Coffin Flop 2. Hot dog car 3. The day the Palins murdered me 4-5 ???
020
NoetherianThing @noetherianthing.bsky.social · 29/09/2026
“President AOC, how many treatments are there?” “At least 3”
160
NoetherianThing @noetherianthing.bsky.social · 28/09/2026
*pagan medieval king voice* if me and my trusty men had been there, Jesus would not have died, but liberty biberty would have…
001
NoetherianThing @noetherianthing.bsky.social · 27/09/2026
Thats an incredible goose playmat....
110
NoetherianThing @noetherianthing.bsky.social · 25/09/2026
We literally can’t train a retriever to fetch without dooming the human species…
010
NoetherianThing @noetherianthing.bsky.social · 25/09/2026
I mean morality in the sense of “oh that’s not what was meant, that’s not How We Do Things”. See the section on filling a cauldron here (linked). The AI can’t be given a simple task without it snowballing to flooding a room. But Claude now would know when to stop: intelligence.org/2016/12/28/a...
intelligence.org
AI Alignment: Why It's Hard, and Where to Start - Machine Intelligence Research Institute
Back in May, I gave a talk at Stanford University for the Symbolic Systems Distinguished Speaker series, titled "The AI Alignment Problem: Why It's Hard, And
141
NoetherianThing @noetherianthing.bsky.social · 25/09/2026
Yeah that’s the gap: 1. Humans are intelligent. 2. Due to intelligence, humans are like utility maximizes. 3. Therefore AI, being intelligent, will be a utility maximizer. 4. Utility maximizes try to paper clip the universe. but humans don’t?!
2121
NoetherianThing @noetherianthing.bsky.social · 25/09/2026
That’s the fault of the excerpting, sorry! He says earlier that the behaviors he means are “penny pumping” like “I’d buy an apple for $1, trade a banana for an apple, and sell a banana for $0.5”. He’s alluding to the von Neumann-morgenstern theorem iirc
150
NoetherianThing @noetherianthing.bsky.social · 25/09/2026
You’ve launched me deep into the archives, but to Yud’s credit, here is a 2016 talk where he specifically says its “as if” the agents have a utility function. (Pic 1) But then he spends the rest of the talk reasoning about specific utility functions so idk (pic 2)
160
NoetherianThing @noetherianthing.bsky.social · 25/09/2026
And after a few more minutes of googling I can say: it was not a Friedman as-if maximizer. The AIs they considered chose the actions that maximized EV on an explicit utility function programmed into them This is core to their theoretical results like convergent instrumentality and incorrigibility
1121
NoetherianThing @noetherianthing.bsky.social · 25/09/2026
I’m unfortunately not familiar with Friedman enough to answer, but I really think their idea was “the AI sits down, reasons every possible outcome from first principles, then chooses the plan for its whole future”. The explicitly defined utility function is there
1110
NoetherianThing @noetherianthing.bsky.social · 25/09/2026
I am worried I fired off the cuff and misremembered Yud, but what do you make of this? From his 2022 AGI: A List of Lethalities
110
NoetherianThing @noetherianthing.bsky.social · 25/09/2026
This mistake is quintessential rationalism because it assumes the AI would be a formally rational utility-maximizer. But LLMs at their core are NOT THAT. They, like human cognition, choose a next step from some opaque internal calculation which often, but not always, leads towards a goal
1455
NoetherianThing @noetherianthing.bsky.social · 25/09/2026
I’m trying to post receipts, and in doing so I should move the Yud school original sin back one step: they assumed all AI would behave by maximizing an explicitly defined utility function, and therefore can provably become squiggle maximizers. Pic from www.lesswrong.com/w/squiggle-m...
Screenshot of lesswrong with this text:

Description

First mentioned by Yudkowsky on the extropian's mailing list, a squiggle maximizer is an artificial general intelligence (AGI) whose goal is to maximize the number of molecular squiggles in its collection. 

Most importantly, however, it would undergo an intelligence explosion: It would work to improve its own intelligence, where "intelligence" is understood in the sense of optimization power, the ability to maximize a reward/utility function—in this case, the number of paperclips. The AGI would improve its intelligence, not because it values more intelligence in its own right, but because more intelligence would help it achieve its goal of accumulating paperclips. Having increased its intelligence, it would produce more paperclips, and also use its enhanced abilities to further self-improve. Continuing this process, it would undergo an intelligence explosion and reach far-above-human levels.

It would innovate better and better techniques to maximize the number of paperclips. At some point, it might transform "first all of earth and then increasing portions of space into paperclip manufacturing facilities".
1316
NoetherianThing @noetherianthing.bsky.social · 25/09/2026
Clarifying “morality is in there”: that’s not to say the AI acts morally, but that it *knows what the “moral” action is* for a given definition of morality
1120
NoetherianThing @noetherianthing.bsky.social · 25/09/2026
The Yud school in particular thought that all significant AI would derive itself from rational first principles, so made a big point that there was no way to communicate morality to AI. But it turns out, like all other intelligences, we trained AI from existing human text, so morality is in there
612813
NoetherianThing @noetherianthing.bsky.social · 24/09/2026
Maybe this is how gyms end up mono-type: mom only has a water type, so she only gets her kids fire types, and 15 years later they’re a fire type gym leader
0241
NoetherianThing @noetherianthing.bsky.social · 23/09/2026
I know... why do they choose to color grade it that way...
110
NoetherianThing @noetherianthing.bsky.social · 22/09/2026
Car driving and murder are different in that people choose the former as part of a rational weighing of options (vs transit, walking, biking, carpool, living closer to work, etc), so a small fee can change the calculation for what’s best. Murder on the other hand is rarely a rational calculation
1120
NoetherianThing @noetherianthing.bsky.social · 22/09/2026
Woke: Obamacare death spiral from healthy young people asking "why do I have insurance if I never use it?" Bespoke: Obamacare death spiral from witch hunts asking "why do you have insurance if you never use it?"
030
NoetherianThing @noetherianthing.bsky.social · 22/09/2026
And just like that, the risk of doom from AI drops to 0!
000
NoetherianThing @noetherianthing.bsky.social · 21/09/2026
I smile at my opponent. Cynthia does not return the gesture. She'd be prettier if she did, I think. Her Garchomp uses earthquake. The move OHKOs my entire team.
0230
NoetherianThing @noetherianthing.bsky.social · 20/09/2026
Was this just E being blocked, or also O and U?
251
NoetherianThing @noetherianthing.bsky.social · 18/09/2026
I’m still holding out for Greta Thunberg’s Narnia movies…
000
NoetherianThing @noetherianthing.bsky.social · 17/09/2026
“Only Democrats have agency” comes for AI…
000
NoetherianThing @noetherianthing.bsky.social · 13/09/2026
Joking!
110
NoetherianThing @noetherianthing.bsky.social · 13/09/2026
Unfortunately for you, the chronology is clear: Trump was first elected in November 2016, and when was Douyin (TikTok) released in China? September 2016. The causation is obvious.
120
NoetherianThing @noetherianthing.bsky.social · 12/09/2026
Left guy could be Thomas Mann!
000
NoetherianThing @noetherianthing.bsky.social · 12/09/2026
Nice try, unfortunately I’ve already depicted myself as the pointy-black-hatted wojak, and you as the no-hat wojak
160
NoetherianThing @noetherianthing.bsky.social · 11/09/2026
Here’s my objection: that 95% CI is valid under the hypothesis that you’re drawing a representative sample of the population. But clearly you’re not, because the individual polls are all pointing in different directions! So in this domain you shouldnt be throwing “classical” CIs.
230
NoetherianThing @noetherianthing.bsky.social · 09/09/2026
Woah, I hope they will copyright a cool name like A4 (the A is for Apple)
030
NoetherianThing @noetherianthing.bsky.social · 09/09/2026
Thesis: Scooby gang has only 2 personalities Antithesis: Their personalities are based on the 5 colleges Synthesis: Their personalities are based on the 5 colleges, which only have 2 personalities between them
050
NoetherianThing @noetherianthing.bsky.social · 09/09/2026
thank Scooby for the snopes fact checkers, I almost fell for this fake news
130
NoetherianThing @noetherianthing.bsky.social · 06/09/2026
Probably not what you're looking for, but I have fond memories of watching BALTHAZAR with my grandmother. He's the worlds best, sexiest, and Frenchest mortician, and every episode he 1) finds a suspicious cause of death and investigates, 2) takes off his shirt, 3) travels by a different cool vehicle
110
NoetherianThing @noetherianthing.bsky.social · 04/09/2026
As foretold...
1110
NoetherianThing @noetherianthing.bsky.social · 18/08/2026
Since a Factorio steel plate is 1/400th of a metric ton, it only takes 228 steel per second to make 18 metric tons per year! That’s about 2000 unbeaconed furnaces, but foundries cut that enormously
010
NoetherianThing @noetherianthing.bsky.social · 15/08/2026
Sorry to tell on you, but the connection is direct line to Zeus -> sends Hermes to Helen of Troy -> Helen knows her actress's phone number
040
NoetherianThing @noetherianthing.bsky.social · 13/08/2026
“You’re right, it’s not just a confirmation vote – it’s a statement.”
010
NoetherianThing @noetherianthing.bsky.social · 09/08/2026
To survive a Take of that magnitude... wow...
020
NoetherianThing @noetherianthing.bsky.social · 09/08/2026
You really think someone would do that? Just go on the internet and say banning incest is eugenics?
130
NoetherianThing @noetherianthing.bsky.social · 09/08/2026
*looking at a linear regression* it was literally trained on data from the set [0,10] \setminus [2,4]. Its mathematically impossible for it to produce new information about x=3
040
NoetherianThing @noetherianthing.bsky.social · 08/08/2026
Not my ideal division of labor in my society, tbh, but absolutely beautiful as a symbol
010
NoetherianThing @noetherianthing.bsky.social · 08/08/2026
One of the great delights of reading the books after only seeing the movies was how Aragorn's kingship is proved by being a great warrior but ALSO being a great healer
110
NoetherianThing @noetherianthing.bsky.social · 06/08/2026
I need a miniseries about the combine subadministrators struggling to hit the water export quotas
070
NoetherianThing @noetherianthing.bsky.social · 05/08/2026
Great thread princess!
140
NoetherianThing @noetherianthing.bsky.social · 04/08/2026
Just like real life!
A picture of the bird of paradise flower
010
NoetherianThing @noetherianthing.bsky.social · 03/08/2026
Don't talk to me or my flightless birds of paradise ever again
110
NoetherianThing @noetherianthing.bsky.social · 31/07/2026
"All right Matt, here's that picture of Guillaume Piketty you requested"
A picture of French historian Guillaume Piketty edited to add red laser eyes
040
NoetherianThing @noetherianthing.bsky.social · 29/07/2026
And I won’t claim meta health is equal to diversity, but if you want to quantify it, that would probably be a good place to start!
010
NoetherianThing @noetherianthing.bsky.social · 29/07/2026
Completely different game, but 3 Card Blind (a magic format) has a meta diversity rating every month! It’s based on results-weighted Simpsons diversity index. Probably could be adapted to Pokemon
110