iiinova — where your digital self keeps evolving

Existential Risk Observatory @xrobservatory.bsky.social · 03/06/2026

Ever more powerful models will lead to an obvious need for ever more regulation. This is the only option to make AI go well. Politicians delivering what this era needs will, in the end, come out on top.

000

Existential Risk Observatory @xrobservatory.bsky.social · 03/06/2026

Also, solutions will need to be found for the non-existential issues AI creates, such as mass unemployment, runaway inequality (both inside and between countries), data center greenhouse gas emissions, and many others. If permanently entrenched, these could become existential in their own right.

100

Existential Risk Observatory @xrobservatory.bsky.social · 03/06/2026

- Regulation will need to be internationally aligned, at the very least with China. - Regulation will need to be enforced. Therefore, AI accelerator GPUs, and the supply chain building them, need to be regulated, too. This should start now, since there is a long delay here!

100

Existential Risk Observatory @xrobservatory.bsky.social · 03/06/2026

- On current trends, open-weight AI can achieve the same capabilities just a few months later, but without functional guardrails. Regulation will need to address this when capabilities grow to a worrying level, without increasing inequality.

100

Existential Risk Observatory @xrobservatory.bsky.social · 03/06/2026

- For safety, the least we need is a licensing regime, where unapproved AI may not be released, or not even created, depending on the danger. Such a licensing regime is explicitly not included in this EO.

100

Existential Risk Observatory @xrobservatory.bsky.social · 03/06/2026

- This EO is voluntary, where we need binding AI regulation. - Dangerous AI capabilities also include persuasion, manipulation, political strategy, weapons acquisition, planning, AI development, awareness, and proliferation. See our takeoverbench.com for data on how fast these risks are growing!

takeoverbench.com

TakeOverBench — AI Safety Benchmarks & Takeover Scenarios

AI is rapidly getting better at using weapons, manipulating, hacking, and carrying out long-term plots against us. We track progress towards AI takeover scenarios.

100

Existential Risk Observatory @xrobservatory.bsky.social · 03/06/2026

Major news that the Trump administration signed an AI Executive Order! That having been said, almost everything remains to be done. A thread.

100

Reposted by Existential Risk Observatory

PauseAI @pauseai.bsky.social · 20/05/2026

As a third of university students in Great Britain think AI job losses will cause social unrest, PauseAI launches a new campaign: AI is not just coming for your job Tell us your story.

131

Existential Risk Observatory @xrobservatory.bsky.social · 14/05/2026

Anthropic are not necessarily the good guys. The existential risk from their models is as high as from those of the other labs. On top of that, they seem to not care at all about climate, that other existential threat.

020

Existential Risk Observatory @xrobservatory.bsky.social · 30/04/2026

Whether left or right, we think it is a core responsibility of democratic politicians to inform the public about risks we are all facing. We applaud @sanders.senate.gov for taking his responsibility seriously.

000

Reposted by Existential Risk Observatory

PauseAI @pauseai.bsky.social · 17/04/2026

Maxime Fournes, Pause AI CEO, addressing MEPs in Brussels. Watch the intervention in full: www.youtube.com/watch?v=aeLz...

052

Reposted by Existential Risk Observatory

PauseAI @pauseai.bsky.social · 13/02/2026

The planet's largest AI summit starts on Monday in India. Will AI safety be on the agenda? Sign our petition to demand that it is. www.change.org/p/ai-summits... #aisafety #aigovernance #artificialintelligence #ai

042

Existential Risk Observatory @xrobservatory.bsky.social · 02/01/2026

This seems like an obvious political chance. It is hopeful that @sanders.senate.gov is on the ball here. We're waiting for others to follow. youtu.be/zJHYVzB4Nu0?...

youtu.be

Sen. Bernie Sanders' AI warning

YouTube video by CNN

010

Existential Risk Observatory @xrobservatory.bsky.social · 02/01/2026

Obviously, we will need to tax AI companies, data centers, and other automated companies, and use this money to provide a high living standard (at least a UBI) for all. It is crucial to set minimum tax rates in international treaties to make sure this is globally achievable.

100

Existential Risk Observatory @xrobservatory.bsky.social · 02/01/2026

If and when we'll get AGI, if we do not directly go extinct, one major problem will be how to divide income. AGI and robotics would likely make us all become unemployed.

100

Reposted by Existential Risk Observatory

PauseAI @pauseai.bsky.social · 02/12/2025

This Christmas, consider funding a PauseAI volunteer.

142

Reposted by Existential Risk Observatory

ControlAI @controlai.com · 29/11/2025

MIRI CEO Malo Bourgon explains why AI isn't like other technologies, and why it looks likely that superintelligence will be developed much earlier than previously thought:

032

Existential Risk Observatory @xrobservatory.bsky.social · 28/11/2025

Xriskers should see the obvious and campaign together with those concerned about data centers, aiming for xrisk awareness raising and getting good regulation implemented.

010

Existential Risk Observatory @xrobservatory.bsky.social · 28/11/2025

AI using water and energy that was made for human beings is an obvious resource conflict, too. There's a continuum straight from these issues to human replacement and eventually human extinction. The more powerful AI gets, the faster this will go.

100

Existential Risk Observatory @xrobservatory.bsky.social · 28/11/2025

Our core concern is humanity getting replaced by AI. Gradual disempowerment is one scenario many worry about. What failure looks like, where factories start sucking up our oxygen, is another. Even the classic paperclip maximizer scenario is a resource conflict at heart.

100

Existential Risk Observatory @xrobservatory.bsky.social · 28/11/2025

Already, these issues are big enough for politicians from left to right to win elections on. Xriskers can read an exponential curve. If this is true today, imagine what AI politics will look like five years from now!

100

Existential Risk Observatory @xrobservatory.bsky.social · 28/11/2025

So far, most xriskers have felt too good for anti-data center campaigning. We made fun of data center water usage and electricity consumption, even though these are actual problems.

semafor.com

View: Trump’s AI agenda sails toward an iceberg of bipartisan populist fury

The AI industry’s new super PAC picked its first political target this month — and missed.

100

Existential Risk Observatory @xrobservatory.bsky.social · 07/11/2025

This trial will be aimed at @stopai.bsky.social, but we all know that Sam Altman is the one doing what should really be illegal. Congratulations to StopAI for making this happen!

000

Existential Risk Observatory @xrobservatory.bsky.social · 07/11/2025

Debating this absurd situation in public is badly needed. It's an even better idea to do so with one of the worst perpetrators, who has time and again tried to build exactly the kind of AI that could kill us all, and who has time and again lobbied hard against any regulation aiming to keep us safe.

100

Existential Risk Observatory @xrobservatory.bsky.social · 07/11/2025

Sometimes, it is hard to believe that this is all real. Are people really building a machine that could be about to kill every living thing on this planet? If this is not true, why are the best scientists in the world saying it is? If this is true, why is no one trying to do anything about it?

100

Existential Risk Observatory @xrobservatory.bsky.social · 23/06/2025

If one in ten experts think there is a risk of human extinction when developing a technology, we should not develop this technology, until we are confident that the risk can be almost ruled out.

000

Existential Risk Observatory @xrobservatory.bsky.social · 18/06/2025

📢 Event coming up in Amsterdam!📢 Many think we should have an AI safety treaty, but how to enforce it?🤔 Riccardo Varenna from TamperSec has part of a solution: sealing hardware within a secure enclosure. Their proto should be ready within three months. Time to hear more! Be there! lu.ma/v2us0gtr

lu.ma

Can a small startup prevent AI loss of control? - with Riccardo Varenna · Luma

According to many leading AI researchers, there is a chance we could lose control over future AI. We think one of the most important challenges of our century…

000

Reposted by Existential Risk Observatory

ControlAI @controlai.com · 11/06/2025

BREAKING: New experiments by former OpenAI researcher Steven Adler find that GPT-4o will prioritize preserving itself over the safety of its users. Adler set up a scenario where the AI believed it was a scuba diving assistant, monitoring user vitals and assisting them with decisions.

111

Existential Risk Observatory @xrobservatory.bsky.social · 11/06/2025

Slowly, but surely, the public is getting informed that there is a level of AI that may kill everyone. And obviously, an informed public is not going to let that happen. Never mind SB1047. In the end, we will win.

100

Existential Risk Observatory @xrobservatory.bsky.social · 11/06/2025

What is interesting is that the presenter assumes familiarity with not only the possibility that AI could cause our extinction, but also the fact that many experts think there is an appreciable chance this may actually happen.

110

Existential Risk Observatory @xrobservatory.bsky.social · 11/06/2025

Two weeks ago, Geoffrey Hinton informed a New Zealand audience that AI could kill their children. The presenter announced the part as: "They call it p(doom), don't they, the probability that AI could wipe us out. On the BBC recently you gave it a 10-20% chance".

110

Existential Risk Observatory @xrobservatory.bsky.social · 03/04/2025

The closer we get to actual AI, the less people like intelligence, however measured. Passing the Turing test is downplayed now, but passing Marcus' Simpsons test will be downplayed later when it happens, too. Still, AI reaching human level is actually important. We can't keep our heads in the sand.

011

Existential Risk Observatory @xrobservatory.bsky.social · 26/03/2025

More info and discussion here: forum.effectivealtruism.org/posts/XJuPEy... www.lesswrong.com/posts/sc4Kh5...

000

Existential Risk Observatory @xrobservatory.bsky.social · 26/03/2025

- Offense/defense balance. Many seem to rely on this balance favoring defense, but so far little work has been done on aiming to determine whether this assumption holds, and in fleshing out what such defense could look like. A follow-up research project could be to shed light on these questions.

100

Existential Risk Observatory @xrobservatory.bsky.social · 26/03/2025

Our follow-up research might include: - Systemic risks, such as gradual disempowerment, geopolitical risks (see e.g. MAIM), mass unemployment, stable extreme inequality, planetary boundaries and climate, and others.

100

Existential Risk Observatory @xrobservatory.bsky.social · 26/03/2025

- Require security and governance audits for developers of models above the threshold. - Impose reporting requirements and Know-Your-Customer requirements on cloud compute providers. - Verify implementation via oversight of the compute supply chain.

100

Existential Risk Observatory @xrobservatory.bsky.social · 26/03/2025

Based on our review, our treaty recommendations are: - Establish a compute threshold above which development should be regulated. - Require “model audits” (evaluations and red-teaming) for models above the threshold.

100

Existential Risk Observatory @xrobservatory.bsky.social · 26/03/2025

Our paper "International Agreements on AI Safety: Review and Recommendations for a Conditional AI Safety Treaty" focuses on risk thresholds, types of international agreement, building scientific consensus, standardisation, auditing, verification and incentivisation. arxiv.org/abs/2503.18956

arxiv.org

International Agreements on AI Safety: Review and Recommendations for a Conditional AI Safety Treaty

The malicious use or malfunction of advanced general-purpose AI (GPAI) poses risks that, according to leading experts, could lead to the 'marginalisation or extinction of humanity.' To address these r...

100

Existential Risk Observatory @xrobservatory.bsky.social · 26/03/2025

New paper out!📜🚀 Many think there should be an AI Safety Treaty, but what should it look like?🤔 Our paper starts with a review of current treaty proposals, and then gives its own Conditional AI Safety Treaty recommendations.

121

Existential Risk Observatory @xrobservatory.bsky.social · 06/03/2025

Richard Sutton has repeatedly argued that human extinction would be the morally right thing to happen, if AIs were smarter than us. Yesterday, he won the Turing Award from @acm.org. Why is arguing for and working towards extinction fine in AI? youtu.be/pD-FWetbvN8&...

youtu.be

Rich Sutton - The Future of AI

YouTube video by UBC Computer Science

010

Existential Risk Observatory @xrobservatory.bsky.social · 06/02/2025

It is hopeful that the British public and British politicians support regulation to mitigate the risk of extinction from AI. Other countries should follow. In the end, a global AI Safety Treaty should be signed.

000

Existential Risk Observatory @xrobservatory.bsky.social · 24/01/2025

On the eve of the AI Action Summit in Paris, we proudly announce our AI Safety Debate with Prof. Yoshua Bengio!📢 In the panel: @billyperrigo.bsky.social from Time @kncukier.bsky.social from The Economist Jaan Tallinn from CSER/FLI Emma Verhoeff from @minbz.bsky.social Join here! lu.ma/g7tpfct0

lu.ma

AI Safety Debate with prof. Yoshua Bengio · Luma

Progress in AI has been stellar and does not seem to slow down. If we continue at this pace, human-level AI with its existential risks may be a reality sooner…

130

Existential Risk Observatory @xrobservatory.bsky.social · 09/01/2025

Pretraining may have hit a wall, but AI progress in general hasn't. Progress in closed-ended domains such as math and programming is obvious, and worrying. The public needs to be kept up to date on both increasing capabilities, and obvious misalignment of leading models.

041

Existential Risk Observatory @xrobservatory.bsky.social · 01/01/2025

Nobel Prize winner Geoffrey Hinton thinks there is a 10-20% chance AI will "wipe us all out" and calls for regulation. Our proposal is to implement a Conditional AI Safety Treaty. Read the details below. www.theguardian.com/technology/2...

theguardian.com

‘Godfather of AI’ shortens odds of the technology wiping out humanity over next 30 years

Geoffrey Hinton says there is 10% to 20% chance AI will lead to human extinction in three decades, as change moves fast

011

Reposted by Existential Risk Observatory

Future of Life Institute @futureoflife.org · 05/12/2024

💼 We're hiring a Head of US Policy! ⬇️ 🇺🇸 This opening is an exciting opportunity to lead and grow our US policy team in its advocacy for forward-thinking AI policy at the state and federal levels. ✍ Apply by Dec. 22 and please share: jobs.lever.co/futureof-life/c933ef39-588f-43a0-bca5-1335822b46a6

023

Existential Risk Observatory @xrobservatory.bsky.social · 25/11/2024

Peaceful activism from organizations such as @pauseai.bsky.social is a good way to increase pressure on governments. They need to accept meaningful AI regulation, such as an international AI safety treaty.

030

Existential Risk Observatory @xrobservatory.bsky.social · 22/11/2024

It is still quite likely AGI will be invented in a relevant timespan, for example the next five to ten years. Therefore, we need to continue informing the public about its existential risks, and we need to continue proposing helpful regulation to policymakers. Our work is just getting started.

000

Existential Risk Observatory @xrobservatory.bsky.social · 22/11/2024

It doesn't appear like we have quite figured out the AGI algorithm yet, despite what Sam Altman might say. But more and more startups, and then academics, and finally everyone, will be in a position to try out their ideas. This is by no means a safer situation.

100

Existential Risk Observatory @xrobservatory.bsky.social · 22/11/2024

So are we back where we started? Not quite. Hardware progress has continued. As can be seen in the graph above, compute is rapidly leaving human brains in the dust. Also, LLMs could well provide a piece of the puzzle, if not everything.

100