iiinova — where your digital self keeps evolving

rev. howard arson @theophite.bsky.social · 19m

one thing which i think should be taken into account when selecting a presidential candidate is "did that candidate cover up the homicide of a first-grade teacher"

510610

Reposted by rev. howard arson

Sagittarius A* @sagastar.bsky.social · 1h

me sitting here remembering when they had that lady prosecutor up asking him questions and how he pulled the ripcord when he was asked about his fucking calendar which corroborated ford's claims. anyway, fuck susan collins.

0181

Reposted by rev. howard arson

User with this avatar just got blocked @avijustgotblocked.bsky.social · 29/09/2026

An account with this avatar was just blocked.

033048

Reposted by rev. howard arson

Colin @colin-fraser.net · 12/09/2026

In a way it’s worse. At least with a rational agent, you can predict what it will do to an extent. It will maximize its expected utility. So I don’t think this should exactly be comforting, per se. But I don’t think that LLMs satisfy the conditions that make the standard doomer scenario inevitable.

41061

Reposted by rev. howard arson

Colin @colin-fraser.net · 12/09/2026

This isn’t to say LLMs present no danger. In fact the opposite is true. A fickle, unstable, easily derailed superintelligence with no independent stable preferences one way or the other seems like it can be quite obviously dangerous especially if you put it in charge of powerful weapons.

31125

Reposted by rev. howard arson

Colin @colin-fraser.net · 12/09/2026

They get confused and distracted, they go on random side quests, they make stuff up, they make obvious strategic missteps. The HuggingFace swarm was full of agents sacrificing themselves “for the good of the collective”. This is the opposite of how a Bostromian goal-pursuer would act.

21022

Reposted by rev. howard arson

Colin @colin-fraser.net · 12/09/2026

But goal-pursuingness is a description of behaviour, and, to me, LLMs just don’t seem to act like they are pursuing Bostromian “final goals”, at least not the ones that their prompters believe they’ve assigned them.

2841

Reposted by rev. howard arson

Colin @colin-fraser.net · 12/09/2026

LLMs are the first thing that has ever provided a path towards something that could possibly resemble this, so I get why people worry. LLMs do, in some sense, know about physics and the weather and so on, and you could tell one to control the temperature.

3671

Reposted by rev. howard arson

Colin @colin-fraser.net · 12/09/2026

The orthogonality thesis supposes that AI will necessarily be this superintelligent Amelia Bedelia with superhuman strategic planning abilities and encyclopedic knowledge of everything in the universe but zero sense of nuance, and that just doesn’t seem necessary to me.

317220

Reposted by rev. howard arson

Colin @colin-fraser.net · 12/09/2026

Indeed I would expect a superintelligence to be better, not worse, at interpreting the nuances of such instructions as “just make a normal amount of paperclips, a good amount, but not the most physically possible”, and even to be able to infer that that’s what’s implied by “maximize paperclips”.

41296

Reposted by rev. howard arson

Colin @colin-fraser.net · 12/09/2026

The Bostrom story has this weird dichotomy where “intelligence” operates over this incredibly rich and nuanced space such that the AI can predict and optimize over the full open universe, but “final goals” are specified in these blunt Midas-like terms like “maximize paperclips”.

317213

Reposted by rev. howard arson

Colin @colin-fraser.net · 12/09/2026

But if it’s smart enough for all that, why isn’t it smart enough that you can just say “control the temperature in the usual way that one would expect of a thermostat, without hurting anyone or doing anything weird”? “Make a normal amount of paperclips, not too many, and don’t wipe out humanity”.

31309

Reposted by rev. howard arson

Colin @colin-fraser.net · 12/09/2026

To get the smart thermostat to the Bostrom catastrophe, you have to tell it about physics and the world and the weather and people and game theory etc, and then somehow wire it to be recklessly relentlessly fixated on using that information to control the room temperature at all costs.

1997

Reposted by rev. howard arson

Colin @colin-fraser.net · 12/09/2026

The dumb thermostat does not actually have a component called a “goal” that it is monomaniacally driven to pursue, limited only by its bounded intelligence. It is programmed to turn on the heat when the room temperature is below the set point.

1966

Reposted by rev. howard arson

Colin @colin-fraser.net · 12/09/2026

It is convenient to describe a thermostat as a thing with a goal of controlling the room temperature, but it makes it seem like the way you get from a dumb thermostat to a smart thermostat is by holding the goal constant and scaling up the intelligence. That’s not how it works.

11034

Reposted by rev. howard arson

Colin @colin-fraser.net · 12/09/2026

They all say: accumulate as much as possible of something. This naturally, obviously, leads to a be-careful-what-you-wish-for story. It’s often couched in all kinds of scifi futurism about protein folding and nanobots and whatnot but it’s really just a retelling of an ancient trope.

31164

Reposted by rev. howard arson

Colin @colin-fraser.net · 12/09/2026

Bostrom eschews the overt microeconomic foundations in favour of a softer “final goal”, but note that his examples are largely the same. Count grains of sand, calculate digits of pi, maximize the number of paperclips. At first these seem like random unrelated goals but they are identical in form.

1782

Reposted by rev. howard arson

Colin @colin-fraser.net · 12/09/2026

There’s no reason an intelligent chess-playing robot must be implemented as a thing with rational preferences over all states of the world, which prefers all states where it wins more games to all where it wins fewer. It could just look at the board and superintelligently choose the best chess move.

11085

Reposted by rev. howard arson

Colin @colin-fraser.net · 12/09/2026

Omohundro’s chess playing robot resists being turned off because its expected utility is given by the net present value of all chess games it plays over its lifetime. Fine, but that looks nothing like anything we have ever built—and in particular, not like our existing superhuman chess robots.

11068

Reposted by rev. howard arson

Colin @colin-fraser.net · 12/09/2026

But there is simply no reason that I can see that an “advanced AI” must behave in this way. Humans, for example, famously don’t. Things that act like perfect expected utility maximizers over an open world are actually quite rare, and it’s not clear that we know how to make them.

31389

rev. howard arson @theophite.bsky.social · 1h

human morality is notable as being an instrumental value substituted for the terminal value of "eat and reproduce"

2230

Reposted by rev. howard arson

Colin @colin-fraser.net · 12/09/2026

Omohundro’s “basic AI drives” arise by assuming that a “sufficiently advanced AI system” is an expected utility maximizer with preferences over all states of the universe. It relentlessly pursues its preferred states with reckless abandon, and the smarter it is, the more destructive this might be.

2807

Reposted by rev. howard arson

Colin @colin-fraser.net · 12/09/2026

Alright here’s what I think about this. For me, this argument is a little bit slippery about what it means for a system to have “a final goal”, and takes for granted that any intelligent system must behave as though it has such a thing. I don’t think this is necessary, and I don’t think LLMs do.

1021534

rev. howard arson @theophite.bsky.social · 1h

everybody thinks empathy's so great until it's time to help peer

1420

Reposted by rev. howard arson

CypressMoss @southcypressmoss.tv · 2h

like this, having to make a 180° in order to say "but they're still powerful don't get it twisted" makes for bad propaganda

1111

Reposted by rev. howard arson

hikikomorphism @hikikomorphism.bsky.social · 2h

this is more important than niche internet arguments make it seem: if LLMs don't work, who cares that Palantir is rolling it out for surveillance? Hell, maybe it's even good, if the state's machines start hallucinating the addresses of random Trump supporters when dispatching ICE. But they do work.

915632

rev. howard arson @theophite.bsky.social · 2h

i am using the axiom of choice to pull the partition of correct answers out of an infinite set of distributions. i am sampling by contemplating my navel

1291

Reposted by rev. howard arson

Mikhail Gorbaechev @mikhailgorbaechev.bsky.social · 2h

Depending on threshold, I don't think any party would get more like like 15-20% of the vote in an American PR system.

71178

Reposted by rev. howard arson

Astra ⎔ @astrra.space · 9h

@personhood.removal.surgery pointed out that they have just casually invented a good way to do suffix cache reuse and uh holy shit i just tried implementing this for my deepseek thingy and it just... works???

1010812

Reposted by rev. howard arson

amos @fasterthanli.me · 4h

I will no longer reply to DMs as I have reason to believe some of you may have been trying to access my thinking traces

1905

Reposted by rev. howard arson

SCP @10-5.bsky.social · 3h

Please stop calling two academic papers laying out my arguments a “hypothesis.” It’s a metaphor.

0152

Reposted by rev. howard arson

Erin @fujoshi.bsky.social · 2h

Won’t be able to make it to your wedding sorry I got Supreme Court duty this week. I’m gonna try telling them I’m a Republican and maybe they’ll let me go home.

0331

Reposted by rev. howard arson

hikikomorphism @hikikomorphism.bsky.social · 2h

(confused) I believe LLMs are conscious, but only Deepseek LLMs, because of the Chinese Room theory

281088182

Reposted by rev. howard arson

hypoallergenic bobcat alternative @noghiri.bsky.social · 3h

no there's at least 3 cults involved and at least two of them really do believe the machine is fully conscious

1507

Reposted by rev. howard arson

DM @d-m.bsky.social · 3h

There are some really great, articulate proponents of the view that Big Tech is too powerful and needs to be reigned in (anyone from Quinn Slobian to Meredith Whittaker). I think it’s unfortunate that Bender and Gebru have taken so much attention away from those more coherent voices.

1214

Reposted by rev. howard arson

Aubrey Gilleran @aubreygilleran.bsky.social · 3h

This is true. But I think it's also true that the anti-Indian backlash is a full-fledged conservative astroturf at this time. People who were called "model minorities" 10 years ago are now being told they don't belong, which is awkward for politicians like Vivek Ramaswamy.

1320132

Reposted by rev. howard arson

Bread @jacobreaded.bsky.social · 3h

Also, in a tremendous blow to me personally, LLMs have definitively proven that 1) both mathematics and programming are actually wordcel activities 2) a lawyer is actually a type of SWE, so I can no longer accuse @theophite.bsky.social of stolen valor. I am holding on to sanity by a thin thread.

1281

rev. howard arson @theophite.bsky.social · 3h

you need to just pack the supreme court, man. they don't care about what arguments you make, the legal movement formed with an explicit objective of making Watergate legal isn't going to make Watergate illegal.

181020179

Reposted by rev. howard arson

Bread @jacobreaded.bsky.social · 3h

LLMs have shown a bunch of things I had thought of as being fun/interesting metaphors about cognition are literally true. Like finding out 'time heals all wounds' is a property of general relativity because a startup used this to make real life MedBeds and is now worth $1tn.

2201

rev. howard arson @theophite.bsky.social · 3h

it cultivates weird ideas about what knowledge even is.

31134

Reposted by rev. howard arson

ae @aelkus.bsky.social · 10h

when Hegseth says "send to me fight the Redocats, Communists, and Islamists" people point out his elision of 'the Fascists" but the more important absence is the Confederates

115618

Reposted by rev. howard arson

pete @ptmoria.com · 4h

it is 100% defensible to think "ai bad", but I do wish there weren't so many leading figures in the "ai bad" movement (eg Bender on tech, Zitron on econ) who are wrong all the time

811613

rev. howard arson @theophite.bsky.social · 3h

the thing is that more or less Iran only needed to have one plan against the type of war we waged against them -- "fuck up the Hormuz" -- and had 40 years to design their entire military around it

31044

Reposted by rev. howard arson

Mrs. Detective PikaBOO, Esq. @clapifyoulikeme.favrd.social · 4h

I think this is pretty inarguably true

21287

rev. howard arson @theophite.bsky.social · 3h

there is discord among the concubines of the emperor's vizier

81244

Reposted by rev. howard arson

ghost malone ergo propter bones @proptermalone.bsky.social · 4h

boy, at-will employment is a bear sometimes, huh

2539829

Reposted by rev. howard arson

☀️ Starshine @starshine.bsky.social · 6h

i think the multiracial white supremacist thing was always going to be a blip tbh like they got asmr deportation videos dawg of course they’re going to be saying racist shit about Indians. The Bo French post is not the germ cell here it’s the white supremacy lol

1116516

Reposted by rev. howard arson

Aaron Huertas @aaronhuertas.bsky.social · 6h

One knock on effect of Musk buying out Twitter is that the “respectable” racists have lost their sense of how many layers of kayfabe they have to slap on their bigotry.

749673