iiinova — where your digital self keeps evolving

There are many others doing good work in this space. Here are a couple of other resources for those looking to help out: • ai-2040.com/supplements/... • jobs.80000hours.org

We’re also interested in collaborations with other researchers and briefing relevant audiences/individuals. Don’t hesitate to reach out: techgov@intelligence.org

111

Technical Governance Researcher

I’ve been very impressed with the work of our team to date, but our team is still relatively small, and there’s so much more we want to do! We’d love more help. Apply here:

jobs.ashbyhq.com

Researcher for the MIRI Technical Governance Team

100

A common question: “when should humanity hit the brakes?” @pbarnett.bsky.social’s recent paper explains pathways by which governments could lose their ability to restrain AI. We should hit the brakes while they still work! intelligence.org/wp-content/uploads/The-Closing-Window-Peter-Barnett.pdf

100

best paper winner from Naci Cankaya about inference verification: Model activations are bit-exact reproducible when enough information is reported, making verification easier. arxiv.org/abs/2606.00279

100

Recent work from Robi Rahman with Sabiha Tajdari on how to verify what chips are doing based on telemetry data. arxiv.org/abs/2606.19262

100

A menu of technical interventions for stopping dangerous AI development and deployment, and discussion of the role these interventions can play in various plans, by @pbarnett.bsky.social, @aaronscher.bsky.social, David Abecassis. arxiv.org/abs/2507.09801

100

The first comprehensive survey I’m aware of on relevant verification mechanisms (though certainly somewhat outdated given its 2024 publication) by @aaronscher.bsky.social and Lisa Thiergart. techgov.intelligence.org/research/mechanisms-to-verify-international-agreements-about-ai-development

100

Malo Bourgon @malo.online · 25/07/2026

Much more is needed to build the technical and governance tools to deliberately pace frontier AI development—from a moderate slowdown to a halt—and many folks are doing good work in this space already. Here’s a sampling of related work from MIRI. 🧵

162

Yeah, some truths remain elusive.

100

Malo Bourgon @malo.online · 25/07/2026

@mobiusdigitalgames.com, total Outer Wilds cultural victory!

100

Malo Bourgon @malo.online · 25/07/2026

Philosophers have spent centuries asking whether all sufficiently rational minds would converge on the same truths. At least when it comes to video games, the answer seems to be a resounding yes.

100

Framing this stuff as sci-fi is an easy rhetorical move but silly IMO. Turing and I.J. Good at the dawn of the field could see the common-sense concern: if you succeed at building something smarter than humans, it might be hard to control. It‘s not just MIRI folks, see the CAIS extinction statement.

130

I mean, hundreds of folks signed the CAIS extinction statement that don't fall into either of those camps. It's not just Daniel and folks at MIRI etc. And from experience, a lot of policymakers on both sides take this seriously, they're just starting to say so publicly.

130

This tracks. In my experience, for a lot of folks the reaction is usually pretty different once they actually sit down and engage with the arguments/evidence directly. My sense is that's basically what happened when Bernie started to look into this stuff.

Malo Bourgon @malo.online · 04/03/2026

Speaking from personal experience, the list of policymakers who take this seriously, on both sides of the aisle, is longer than most people realize and growing. Grateful to @sanders.senate.gov for being one of them.

161

We’re honored to have had the opportunity to host you at MIRI (@intelligence.org). Thanks for taking the time to chat with us (and many others during your trip to the Bay Area) and raising awareness about the threat we all face from the reckless race to superintelligence.

080

Malo Bourgon @malo.online · 16/02/2026

I was on ABC News Live last week! My first live interview. I think it went pretty well, especially considering I only had ~20 mins notice :p

youtube.com

Top AI researcher warns 'world is in peril'

YouTube video by ABC News

052

Malo Bourgon @malo.online · 01/01/2026

We’re so close! Very grateful for all our donors. Your support enables everything we do. Also grateful for the awesome gang at MIRI who worked their asses off this year. You guys crushed it! Thanks everyone 🙏 Happy New Year 🎉

020

Reposted by Malo Bourgon

MIRI @intelligence.org · 01/01/2026

Final Update: From ~$450k earlier today, we’re now down to just over $250k left in unclaimed matching funds! 4 hours left to go, and by golly it looks like we’ve got a real shot at securing all the matching. Thanks everyone! Happy New Year 🎉

012

Reposted by Malo Bourgon

MIRI @intelligence.org · 31/12/2025

Update 2: We’re down to ~$450k left of unclaimed matching funds, with just over 12 hours to go! Thanks to all those who stepped up in the last couple of days to close the gap by ~$500k. ❤️

011

Reposted by Malo Bourgon

MIRI @intelligence.org · 30/12/2025

Update: We’ve received over $250k since this was posted. ~$700k in matching funds remaining.

032

Reposted by Malo Bourgon

MIRI @intelligence.org · 29/12/2025

Donations to MIRI before Jan 1 are high-leverage. We’ve got ~$1.6M in 1:1 matching from SFF, over half of which has yet to be claimed! This is real counterfactual matching: whatever doesn’t get matched by the end of Dec 31, we don’t get. 🧵

intelligence.org

MIRI's 2025 Fundraiser - Machine Intelligence Research Institute

MIRI is running its first fundraiser in six years, targeting $6M. The first $1.6M raised will be matched 1:1 via an SFF grant. Fundraiser ends at midnight on Dec 31, 2025. Support our efforts to impro...

132

You don't have to take my word for it. LLMs are dumb in a bunch of ways, but I think this is a powerful and convincing consensus on this question.

000

Seems like you’re confusing a punchy post title about a standard norm in Bayesian epistemology (i.e., don’t give empirical claims credence 0 or 1, or you can’t update) with a claim about the formal definition of probability, where 0 and 1 are of course valid probabilities.

100

Of course, by definition probabilities are real numbers on [0,1], which includes the endpoints.

100

Malo Bourgon @malo.online · 04/12/2025

I understand that you believe I’m a huckster. I was hoping you might elaborate on why you think that.

100

Huckster? Say more. Conversations I was having in the comments seemed pretty reasonable and polite, with me just sharing context/info/perspective, and folks following up on that.

100

Malo Bourgon @malo.online · 04/12/2025

... and banned almost immediately. ¯\_(ツ)_/¯

100

Reposted by Malo Bourgon

MIRI @intelligence.org · 31/10/2025

”If Anyone Builds It, Everyone Dies” was recently added to the New Yorker's “The Best Books of the Year So Far” list! newyorker.com/best-books-2...

052

Malo Bourgon @malo.online · 30/10/2025

@hankgreen.bsky.social rarely does interviews or 30+ min long videos. His latest video, an hour+ long interview with Nate Soares about “If Anyone Builds It, Everyone Dies,” is a banger. My new favorite! www.youtube.com/watch?v=5CKu...

061

Reposted by Malo Bourgon

MIRI @intelligence.org · 24/10/2025

In the Bay Area? Come join Nate Soares, in conversation with Semafor Tech Editor Reed Albergotti, about Nate's NYT bestselling book “If Anyone Builds It, Everyone Dies.” 🗓️ Tuesday Oct 28 @ 7:30pm at Manny’s in SF. Get your tickets:

eventbrite.com

Nate Soares - If Anyone Builds It, Everyone Dies

Nate Soares discusses the scramble to create superhuman AI that has us on a path to extinction. But it’s not too late to change course.

141

Malo Bourgon @malo.online · 17/10/2025

IDK man, feels like you are fixated on one particular thing he said (and your interpretation of what he meant by it), as part of a longer conversation on the pod about the analogy. I’m not trying to pick a fight here, just wanted to clarify that he’s not making the mistake you think he’s making.

Malo Bourgon @malo.online · 17/10/2025

I listened to it. Also, I run org (@intelligence.org) that he founded, so I’m quite familiar with the argument he’s making. This interview didn’t result in the best exposition of the analogy in question, but I can assure you he isn’t making the mistake you think he is.

110

Malo Bourgon @malo.online · 17/10/2025

How is he anthropomorphizing natural selection? One can think of evolution as an optimization process, and the analogy is between that optimization process and the one used to train AI systems.

100

Malo Bourgon @malo.online · 16/10/2025

It is by no means a "nearly unanimous view" that LLMs are a dead end among AI experts. Also the argument for future very powerful AI systems being an extinction threat does not depend on those systems being LLM based.

030

Reposted by Malo Bourgon

MIRI @intelligence.org · 15/10/2025

😮 Whoopie Goldberg recommends “If Anyone Builds It, Everyone Dies” on The View!

132

Reposted by Malo Bourgon

Malo Bourgon @malo.online · 18/09/2025

Should be a fun conversation. I'll be there, if you're in town come say hi!

011

Malo Bourgon @malo.online · 23/09/2025

This was a great event. Really enjoyed chatting with Joel and Ollie on the first panel. Thanks @scientistsorg.bsky.social and @futureoflife.org for putting this event together.

041

Malo Bourgon @malo.online · 22/09/2025

(Totally above board. Sharing full-length episodes is one of the benefits of being a subscriber.)