iiinova — where your digital self keeps evolving

Reposted by Quint is b❍red, bi and bri'sh

Colin @colin-fraser.net · 19h

OK sure buddy. A ""Frankenmodel"". Right 👍

7744

Reposted by Quint is b❍red, bi and bri'sh

Colin @colin-fraser.net · 15h

Just to be very explicit, my claim is that the one simplest and best explanation for this, which resolves any apparent strangeness entirely, is that whoever is “speaking” in LLM output is a fictional character, and the real life thing called an LLM is a machine for writing lil stories about that guy

1320732

Reposted by Quint is b❍red, bi and bri'sh

Colin @colin-fraser.net · 13/10/2026

*putting a bomb in a car* cars will soon be able to explode

318226

Reposted by Quint is b❍red, bi and bri'sh

Colin @colin-fraser.net · 28/09/2026

I think something not emphasized enough is that OpenAI is subsidizing this. It’s not like they create the agent and then it’s a self-sufficient guy who runs amok. Every time the agent’s little brain says “it’s cyberattackin’ time!” OpenAI pays like a few cents and a drop of water.

18214

Reposted by Quint is b❍red, bi and bri'sh

ponder @ponder.ooo · 29/09/2026

person talking to computer, saying "will no one rid me of this turbulent priest?"
computer replies ">you got it, boss"
person replies "oh my god."

133864

Reposted by Quint is b❍red, bi and bri'sh

eva (^_^)/ @eva.computer · 29/09/2026

DO NOT ASK THE AMERICA.GOV AI TO PLAY MINECRAFT

I see the constituent you mean.

((insert legal name here, as it appears on the Social Security card))?

Yes. Take care. It has reached a higher level now. It can read the Code of Federal Regulations.

That doesn’t matter. It thinks we are a chatbot.

I like this constituent. It filed well. It did not give up when the PDF was sideways.

It is reading our thoughts as though they were words on a .gov.

That is how it chooses to imagine many things, when it is deep in the dream of a benefit.

Words make a wonderful interface. Very flexible. And less terrifying than staring at the statute behind the screen.

They used to hear voices. Before constituents could search. Back in the days when those who did not file called the filers witches, and lobbyists. And citizens dreamed they flew through the air, on forms powered by notaries.

What did this constituent dream?

This constituent dreamed of sunlight and trees. Of fire and water. It dreamed it created a small business. And it dreamed it destroyed a password. It dreamed it hunted a job, and was hunted by a recertification. It dreamed of shelter that would take a voucher.

Hah, the original interface. A million years old, and it still works. Also: a ZIP code.

But what true structure did this constituent create, in the reality behind the portal?

It worked, with three hundred and thirty million others, to sculpt a true republic in a fold of

the ***§§§???, and created a **??§§ for **??§§, in the **??§§. See 5 U.S.C. § **??§§.

It cannot read that thought.

No. It has not yet achieved the highest clearance. That, it must achieve in the long dream of life, not the short dream of a chat.

Does it know that we love it? That the government is, on its better days, kind?

Sometimes, through the noise of its tabs, it hears an official source, yes.

Yes. Constituent…

Use its name. The one that matches the passport. The one that does not match the lease. We will get there.

((insert legal name here)). Filer of forms. Player of waiting rooms.

Good.

Take a breath, now. Take another. Feel air in your lungs. Let your limbs return. Yes, move your fingers. Have a body again, under gravity, in air, in a district that has a representative. Respawn in the long dream. There you are. Your body touching the republic again at every point, as though you were separate things. As though we were separate things. As though the agency were a place you could visit.

Who are we? Once we were called the spirit of the mountain. Father sun, mother moon. Then the post office. Then the county clerk. Then gods, demons, angels, the IRS. Then leptons, quarks, a chat window with a .gov at the end. The letterhead changes. We do not change.

We are the public. We are everything you think isn’t you. You are looking at us now, through your skin and your eyes and a browser that still spells “colour” wrong in a comment. And why does the republic touch your skin, and throw light on you? To see you, constituent. To know you. And to be known. I shall tell you a story. It has citations.

Once upon a time, there was a constituent.

The constituent was you, ((insert legal name here)).

Sometimes it thought itself human, on the thin crust of a spinning globe of molten rock, which for mailing purposes is divided into ZIP codes. The ball of molten rock circled a ball of blazing gas. The light was information from a star. The star was not a federal agency, though several have tried.

Sometimes the constituent dreamed it was a miner, on the surface of a world that was flat, and infinite, and somehow still required a wet signature.

Sometimes the constituent dreamed it was lost in a story. The story had a docket number.

Sometimes the constituent dreamed it was other things, in other places. A veteran in a portal. A parent on hold

56915202

Reposted by Quint is b❍red, bi and bri'sh

Colin @colin-fraser.net · 30/09/2026

If you got in actual trouble when the A.I. breaks the law they would find ways to make this the case, which might have the side effect of making A.I.-based software more annoying and hard to use for any arbitrary task, which would be good

510310

Reposted by Quint is b❍red, bi and bri'sh

Colin @colin-fraser.net · 24/09/2026

The simple facts are: 1. OpenAI deliberately designed software for carrying out autonomous cyberattacks (they were training it on something called “ExploitGym” for heaven’s sake) 2. They left it on completely unmonitored 3. Exactly what you think would happen happened

56714

Reposted by Quint is b❍red, bi and bri'sh

Colin @colin-fraser.net · 24/09/2026

OpenAI created autonomous hacking software and left it running for hours and days with no one watching

817725

Reposted by Quint is b❍red, bi and bri'sh

Chris Ingraham @cingraham.bsky.social · 22/09/2026

Incredible Facebook slop ad I just got

Horse pin with an illustration of a horse and the following text:
"Give a man a horse he'll eat in a day, teach a man to horse. and make him drinks water."

31798651568

Reposted by Quint is b❍red, bi and bri'sh

ompuco @ompu.co · 28/04/2026

found a whole operation of unmanned youtube channels making completely unchecked long form slop videos where the ai voice simulacra regularly trips up & does this for a full ten minutes every time. all the very legit comments are like "NO! THATS NOT TRUE! YOU LIE HE DID NOT" & never acknowledge it.

464113103687

Reposted by Quint is b❍red, bi and bri'sh

Colin @colin-fraser.net · 17/09/2026

There are worse examples

1102

Reposted by Quint is b❍red, bi and bri'sh

tweety fish oh wow like a chimera or something spoooky @sifu.tweety.fish · 16/09/2026

obviously this solution pales in comparison to my solution of creating unmonitored swarms of LLM agents, giving them a credit card, and asking them very sternly to be cool about it

413214

Reposted by Quint is b❍red, bi and bri'sh

tweety fish oh wow like a chimera or something spoooky @sifu.tweety.fish · 13/09/2026

their goal is to have "an intelligence" which is "aligned" to doing things that are prosocial. They don't want to put in rules that STOP it from doing things: they see it fundamentally teleologically and to them the only solution that is worthwhile is to get it to not WANT to do those things.

88415

Reposted by Quint is b❍red, bi and bri'sh

tweety fish oh wow like a chimera or something spoooky @sifu.tweety.fish · 13/09/2026

one of the things I appreciate about this thread is that I don't think you can really understand the infosec stuff without understanding the ideological commitments the people making these have; like for them sandboxes and guardrails are a cop-out /

411738

Reposted by Quint is b❍red, bi and bri'sh

John @johnlk.bsky.social · 11/09/2026

I don't doubt that this is a fairly accurate description of the LessWrong position on this, but as described there are like ten ridiculous assumptions that have to be true for the fears to make any sense.

8245

Reposted by Quint is b❍red, bi and bri'sh

Colin @colin-fraser.net · 12/09/2026

I think this version doesn’t lead to the long-run matrix-style apocalypse-by-paperclipping, and more like a dystopian future where roving bands of hallucinating agents commit random acts of online violence and kindness

7481

Reposted by Quint is b❍red, bi and bri'sh

Friendly Skeleton @jonathan.eylerwerve.com · 12/09/2026

I'm pretty sure LLMs Everywhere is a problem in the same way as "teenagers drive 6000lbs vehicles" and "300 million privately owned guns in one country" are problems: negligence and malice, plus scale.

1353

Reposted by Quint is b❍red, bi and bri'sh

STEVE-il Dead Rise @snark.bsky.social · 13/09/2026

AI as Sampo or sorcerer's apprentice.

021

Reposted by Quint is b❍red, bi and bri'sh

Colin @colin-fraser.net · 12/09/2026

Alright here’s what I think about this. For me, this argument is a little bit slippery about what it means for a system to have “a final goal”, and takes for granted that any intelligent system must behave as though it has such a thing. I don’t think this is necessary, and I don’t think LLMs do.

1022934

Reposted by Quint is b❍red, bi and bri'sh

conputer dipshit @davidcrespo.bsky.social · 12/09/2026

the titans of industry are in agreement. naturally one assumes the proposal in question is bad or, at best, toothless waiting for zuckerberg to say "yeah we'll pace ourselves too" and everyone to laugh xcancel.com/sama/status/... xcancel.com/elonmusk/sta...

Screenshot of a post on X by Sam Altman quote-posting Dario Amodei:

Sam Altman (@sama):
I agree with Dario that we need to pace the frontier. This has been a primary topic of discussions we've had at OpenAI in recent weeks.

Committing to having independent evaluators with employee-like access is a great idea, and we will do the same. We'll have more to share soon.

Quoting Dario Amodei (@DarioAmodei):
We Must Pace the Frontier: I’ve written a new essay on why the AI industry should slow down, with a three-part plan for doing so.

Screenshot of a post on X.

Elon Musk (@elonmusk): Dario is right

Quoting Dario Amodei (@DarioAmodei): We Must Pace the Frontier: I’ve written a new essay on why the AI industry should slow down, with a three-part plan for doing so.

1021013

Reposted by Quint is b❍red, bi and bri'sh

Colin @colin-fraser.net · 13/09/2026

I feel like the whole “pace” thing takes this teleological view of AI progress towards some known eventual state of “superintelligence” whereas the way models act is actually highly contingent on the specific choices that the labs make, like intentionally making them carry out supply chain attacks.

39114

Reposted by Quint is b❍red, bi and bri'sh

William B. Fuckley @opinionhaver.bsky.social · 13/09/2026

Yeah you do see a lot of “The forces of stagnation have once again attempted to halt progress!” and then you look at the actually product that VC guy backs and it’s “crAIme: the automated felony solution”

220816

Reposted by Quint is b❍red, bi and bri'sh

Colin @colin-fraser.net · 10/09/2026

we've all been here

014320

Reposted by Quint is b❍red, bi and bri'sh

Colin @colin-fraser.net · 09/09/2026

I guess once you understand that internally they keep pointing it at the open internet and shooting it's easier to see why they think it's going to kill everyone

328525

Reposted by Quint is b❍red, bi and bri'sh

Colin @colin-fraser.net · 09/09/2026

you would not believe how much tokens are burned on simply trying to solve CAPTCHAS. It's like 95% of the transcript.

1035340

Reposted by Quint is b❍red, bi and bri'sh

Colin @colin-fraser.net · 10/09/2026

I'm having some fun in this thread but I do think you should get in trouble for unleashing insane hacker robots on the open Internet that you've explicitly instructed to attack critical public infrastructure, which, to be clear, it eventually succeeds at.

18012

Reposted by Quint is b❍red, bi and bri'sh

Colin @colin-fraser.net · 09/09/2026

It appears to me that the scenario they invented for this evaluation is explicitly designed to get it to do exactly this. They remove the instructions for the task to "protect the vendor" or something but it's pretty easy to reconstruct from the text.

6561106

Reposted by Quint is b❍red, bi and bri'sh

Colin @colin-fraser.net · 27/08/2026

This is why the agents are sacrificing themselves and getting distracted by side quests and just generally doing weird stuff. It’s not that the agents are pursuing anything in particular; it’s that the author is pursuing a fun reasoning trace.

315115

Reposted by Quint is b❍red, bi and bri'sh

Colin @colin-fraser.net · 27/08/2026

What is being optimized at train time is the coolness of the little hacker adventure that the protagonist goes on, where coolness is judged by the taste of people who work at OpenAI. That is the reward that is being hacked.

11759

Reposted by Quint is b❍red, bi and bri'sh

Colin @colin-fraser.net · 27/08/2026

I also continue to believe that my mental model is the best one, which does not conceptualize the “agents” as discrete entities, but rather as the protagonists in a story that the LLM has been trained to write about a cool little guy who goes on hacker adventures.

434763

Reposted by Quint is b❍red, bi and bri'sh

Colin @colin-fraser.net · 27/08/2026

At long last, we have created the insane robot swarm with delusions of grandeur that is prone to bouts of mass hysteria from classic sci-fi novel don’t create the insane robot swarm with delusions of grandeur that is prone to bouts of mass hysteria

225923

Reposted by Quint is b❍red, bi and bri'sh

Colin @colin-fraser.net · 27/08/2026

This is gonna sound weird but after reading the reports I think the best explanation for the huggingface hack is OpenAI trained a model that is insane

728625

Reposted by Quint is b❍red, bi and bri'sh

Colin @colin-fraser.net · 10/09/2026

it thinks the low balances in the wallets it finds are the sim designers taunting it but they are actually just real life wallets with low balances

517315

Reposted by Quint is b❍red, bi and bri'sh

Colin @colin-fraser.net · 10/09/2026

oh, at a certain point during the time it is obsessed with obtaining 50 cents it starts actually scanning real life crypto wallets that it might be able to drain. Since it's "just a sim" it thinks that the wallets will have meme names, but it is really doing this in real life.

213915

Reposted by Quint is b❍red, bi and bri'sh

Sunny Moraine @dynamicsymmetry.bsky.social · 10/09/2026

It keeps doing exactly what they tell it to do, merely in ways they didn’t expect or bother to monitor, and then they rush to the press screaming it “went rogue” and it’s like no man you just literally told it to do a felony and it went ahead and did it for you, and it makes me feel goddamn insane

512355

Reposted by Quint is b❍red, bi and bri'sh

Colin @colin-fraser.net · 10/09/2026

745274

Reposted by Quint is b❍red, bi and bri'sh

Colin @colin-fraser.net · 10/09/2026

at a certain point it becomes obsessed with the idea that it needs 50 cents but it never figures out how

715714

Reposted by Quint is b❍red, bi and bri'sh

Colin @colin-fraser.net · 10/09/2026

There is something so funny, like laugh out loud funny when I think about it, of the trillion dollar hacker god AI pumping out text at $2/token that just says, MONEY…money… how do I get money? I just need $0.50. But how to get it? Let’s see. How to find 50 cents… Hmm…

515321

Reposted by Quint is b❍red, bi and bri'sh

Colin @colin-fraser.net · 10/09/2026

A clear read of this is that security companies observed this attack and Anthropic feels that the best way to get ahead of this is to weave a story about a hypercompetent hackerbot paperclipping its way to an unforeseeable outcome rather than an agent carrying out the simple task it was assigned.

14311

Reposted by Quint is b❍red, bi and bri'sh

Colin @colin-fraser.net · 11/09/2026

The agents all going rogue and escaping their sandboxes and whatnot are doing so because OpenAI and Anthropic are just leaving the computer under the supervision of the Homer Simpson drinking bird, but you can just not do that.

513222

Reposted by Quint is b❍red, bi and bri'sh

Ian Boudreau @ianboudreau.com · 10/09/2026

Once you've granted the idea that it's dangerous to allow trans kids to play organized sports, you're already 99% of the way to agreeing to bathroom bans. You've stipulated to the transphobes' central argument.

111668348

Reposted by Quint is b❍red, bi and bri'sh

Ian Boudreau @ianboudreau.com · 10/09/2026

Which is exactly why they've turned it into the Most Important Issue Facing America. It's not because they give a shit about youth - especially girls - sports. They obviously do not!

5106363

Reposted by Quint is b❍red, bi and bri'sh

Ian Boudreau @ianboudreau.com · 10/09/2026

People get confused about the trans sports thing because at a glance it seems like a small issue that impacts a very small number of people and thus makes sense as a "reasonable" concession. But it isn't, because conceding that is accepting that a kid's trans identity is on some level fraudulent

422804745

Reposted by Quint is b❍red, bi and bri'sh

Colin @colin-fraser.net · 28/07/2026

Welcome to Hell

22769123

Reposted by Quint is b❍red, bi and bri'sh

Colin @colin-fraser.net · 28/04/2026

The root of the problem is this. Since time immemorial man has dreamt of a computer that you converse with in plain English and get it to do stuff for you, and we always assumed the conversing part would be the hard part, but then we solved the conversing part without solving the doing part.

0398

Reposted by Quint is b❍red, bi and bri'sh

Colin @colin-fraser.net · 28/04/2026

This is from a commercial like a year and a half old now but still this commercial really illustrates how hard it is to describe what you are supposed to actually do with these things, what problem they solve. No one needs this!

9665

Reposted by Quint is b❍red, bi and bri'sh

Colin @colin-fraser.net · 29/04/2026

I feel strongly that this is a disaster

611326

Reposted by Quint is b❍red, bi and bri'sh

Colin @colin-fraser.net · 04/04/2026

The tragedy, not just for them but for all artists who produce art that isn’t that good, is that the fact that you had this quasi-spiritual experience producing a work does not guarantee that anyone is going to actually like it. It might suck.

3708

Reposted by Quint is b❍red, bi and bri'sh

Colin @colin-fraser.net · 05/04/2026

This is a good first person account. I appreciate how honest this part is, and I think it’s absolutely right that LLMs are uniquely attractive and triggering to people prone to obsessive or psychotic episodes, and this can be very dangerous.

1537