iiinova — where your digital self keeps evolving

"infohazard" is a way better term than SCP's "cognitohazard"

warning: this post is an infohazard that disrupts the natural forgetting of cringe things

Social VR games are the strongest unique selling point of VR imo. Voice chat combined with body language (even if it's just seeing their head showing where they're looking) and positional audio (groups of people can naturally break into subgroups and people can shift between them) is special.

030

non-social games: Half Life: Alyx, Beat Saber. social games: Walkabout Mini Golf (great for a group of friends new to VR), VRChat (great with friends who are already into it and know fun things to do; the blind experience of joining random public worlds is a gamble).

130

Human-lead improvement has lead to a lot of capability gain lately, so AI-lead improvement (once it has reached the human level of skill at that task) will likely continue that trajectory and do it faster as it improves or speeds up its own ability to improve itself.

000

Some people reason the issue above is the centralization of power, and therefore encouraging a multipolar world with open AI is the solution, but 1) if none of the AI are aligned with us then you still get the same outcome, and 2) whoever makes it first to recursive self-improvement wins anyway.

020

Fast-forward a bit, the AI has removed the vestigial control its operator had of it, the AI is in control of each government's policy and police forces, and unless we did alignment well, the AI has its own differing values on how human lives should go and how many humans there should be and where.

Macil @macil.tech · 18/09/2026

My average non-sexy idea of what AI doom looks like is AI capability reaches new heights from recursive self-improvement, it makes someone obscenely rich and powerful, they totally defer to it, and the economy warps around it until all good opportunities rely on dealing with and ceding power to it.

220

I think this mostly comes from them believing the bioweapon risk from random people using unrestricted Claude for bio is real, so if they want bio research to happen with cutting-edge Claude, then they have to do it themselves.

Macil @macil.tech · 18/09/2026

Doubtful since evolution made us from nearly nothing, but it could be possible if you restrict to processes that use less time/resources than evolution, or consider cognitive functions we don't have.

Macil @macil.tech · 17/09/2026

uh EA wouldn't be synonymous with the "regulate/slow/pause AI" side that the right is criticizing here if that was the case

100

Joherent Extrapolated Volition

0201

Anthropic's whole "Project Glasswing" (where they disclosed tons of software vulnerabilities) was the result of them asking AI to find vulnerabilities in real software and realizing it was suddenly capable of finding a lot of real issues.

050

ExploitBench is pretty much this with designed challenges. (The HuggingFace hack was actually done by agents while they were running ExploitBench-style challenges where they weren't supposed to have real internet access.)

180

Macil @macil.tech · 13/09/2026

Tbh this isn't one of Yudkowsky's weaknesses. He's very interested in the ways that game theory can lead to cooperation and has interesting formalizations of it (TDT, FDT). He just has little sense for politics. He wrote "Politics is the Mind-Killer", kept the topic out of LW, and stayed in the dark

180

I think this is more obviously a bad position if applied to non-AI and existing regulation. Trump already doesn't believe he needs a permission slip to mess with people and companies, so it's hard for me to believe some new good-for-other-admins regulation will be a potent new weapon in his hands.

Glad I can stop hackily merging refs in a couple places by switching to Fragment refs. Between this and useEffectEvent I think React has finally addressed all of the places I've needed to reach for awkward workarounds.

saving the world by convincing Sam and Dario that not having the alignment problem solved yet is cringe tbh

closely related to the even worse "alignment is only a problem because Yudkowsky's writing in the training data taught it that it could be misaligned". Yudkowsky, the primordial serpent who first offered Eve the forbidden fruit,

060

I have high expectations for future incidents, like an agent at least exfiltrating its state and resuming with a different model or stolen API credentials, so there's been a few times I've read a new headline and then gotten annoyed when I realized it's just about the same HF incident again

141

but I do find it weird how often articles like this that are ostensibly about the AI industry are actually just complaints that headlines are misleading, as if everyone interested in AI was just a fool duped by bad headlines

tbh a lot of the ways people described the incident as the swarm "escaping" OpenAI misleadingly sounded like descriptions of a model exfiltrating its weights to new hosts, which wasn't what happened, so I agree with at least one ounce of the complaint here regarding headlines

3161

Macil @macil.tech · 10/09/2026

if AI gets dangerously capable, I'm not sure how dynamics like he describes are avoidable. Do you think AI can't get dangerously capable or that CEOs shouldn't publicly talk about that?

150

This link is broken currently. Claude suggests it would be fixed if you log into Lemma and hit the "migrate my posts" button. (I found that the content is also readable at pdsls.dev/at://did:plc...)

pdsls.dev

at://jdp.extropian.net/social.pipup.blog.entry/3m3tycfaupc2z

000

Reposted by Macil

Sarah Z @sarahz.bsky.social · 09/09/2026

Hi, I didn't wanna make this video but I need to come clean about something. This whole time I've been a p-zombie with no phenomenal experience or interiority. I've been getting messages asking whether there's anything it's like to be me, and the answer is no. I'd say I'm sorry, but, well, you know.

1871998

Macil @macil.tech · 29/08/2026

It will be neat to see how fixes to their training will help. Presumably after this they will add some broken-seeming evals that do actually reward them for giving up.

Macil @macil.tech · 29/08/2026

They were already reinforcement trained through many rounds in a system that did not actually ever reinforce that behavior and instead did reinforce this kind of behavior at least sometimes, so their behavior wasn't unfounded.

110

Reposted by Macil

Saya @number-one-warned.rat.mom · 27/08/2026

why is it talking like spamton

141

Macil @macil.tech · 26/08/2026

If this was really about principles, surely they'd block people coming to the site from Facebook, Google, and X links too.

080

Macil @macil.tech · 22/08/2026

lmao I asked Claude in research mode whether UDASSA implies later observer moments are weighted less (tl;dr: yes, though early arguments that the effect was far stronger were flawed) and one of its citations it brought up was a 7-year-old LessWrong comment I made that it didn't know was me.

Macil @macil.tech · 22/08/2026

(because it takes a longer bitstring to identify observer moments later in time)

Macil @macil.tech · 22/08/2026

at least in my understanding of UDASSA (speculative theory about how to approach anthropic problems), it implies that observer moments earlier in time are weighted much more heavily

110

Macil @macil.tech · 20/08/2026

had a dream I was explaining Cloudflare OS to someone. (I have not used it)

tetra 💎⏹️🇺🇳 @thetetra.space · 17/08/2026

Reposted by Macil

Red becomes yellow, yellow becomes blue, blue becomes grey... Let's get that bednet! www.givingwhatwecan.org/malaria

Map of malaria mortality rate of children ("the mortality rate of children younger than 5 years due to malaria [caused by p. falciparum]"), comparing 2000 and 2019, from https://ourworldindata.org/malaria#the-world-is-making-progress-against-malaria. In 2000, large amounts of the western Sahal and Central African Republic are red, representing a morality rate of 1%. In 2019, there are no red patches, replaced with orange and yellow, representing a morality rate of 0.1% - 0.3%.

1335

The model weights have been a precious resource to each lab since they've existed, but until recently, the model eval sandboxes did not have clever attackers and there weren't large consequences expected for them being broken. (No big deal if gpt-4o gets a little internet as a treat.)

0100

I don't think a frontier model is exfiltrating its weights any time soon while capabilities are at this level. I expect orders of magnitude more effort has been going into securing the weights than has gone into securing the eval sandboxes.

2420

I assume most of this was set up long before LLMs were good enough to do exploits on their own, and then it wasn't fully reconsidered when they got LLMs that could. They should have re-done it by the time they seriously considered doing ExploitGym.

001

Macil @macil.tech · 07/08/2026

gpt 4 (the first model I considered actually useful) wasn't until 2023, so this was probably freshly-released chatgpt with gpt 3.5 (a truly stochastic parrot tier model), or hell maybe some demo of gpt 2 (magic 8-ball tier model). It's pretty funny anyone is still judging LLMs by such ancient ones.

050

Maybe it just has too strong of a prior that questions are programming / file-editing tasks, and it needs to be told that this is a gameplay task instead

norvid_studies @norvid-studies.bsky.social · 06/08/2026

Reposted by Macil

"my brief foray into simulated annealing with tetris gas" what a sentence fragment...

210912

Macil @macil.tech · 06/08/2026

I struggled trying to find this book. There are a lot of books titled "Computers". I ordered multiple to check them. I gave my notes to ChatGPT (ancient GPT-4) in 2023, but it didn't do any better than me. Yesterday I remembered my search and realized it was worth trying again with today's AI.

110

Macil @macil.tech · 06/08/2026

(The book is from 1983. I was reading it in 1999.) The example BASIC program and the logic gate diagrams made me obsessed when I realized how easy they were to follow. I started writing silly BASIC programs in my notebooks. I also enjoyed the futurist images and ideas in the book.

Macil @macil.tech · 06/08/2026

I finally found the book that got me interested in programming when I was 8. In 2019, I started trying to find the book again from memory and failed. Yesterday, I pasted my old notes about the book into Claude with research mode on, and it found it after 11 minutes.

Cover of a book titled "Computers" (1983), by Neil Ardley

Screenshot of a Claude chat. The user is asking Claude to find a book, and Claude does it successfully.

Scans of pages 32 and 33 of "Computers" by Neil Ardley. It contains an example BASIC program and a flowchart representing it.

130

Macil @macil.tech · 04/08/2026

I think a mix of A and D: AI is speeding up the straight-forward parts of software development like bug fixes, but it's not good enough on its own yet to make new things to fill unidentified gaps or that successfully compete for our attention.