The catastrophic shutdown problem is (roughly) the problem of designing AI systems that can be shut down when their acts would lead to existential catastrophe.

The case for longtermism is strongest on fanatical decision theories. They might be wrong.

Yudkowsky and Soares have a simple solution to AI safety: Shut it down.

Effective altruists often go to great lengths to engage with their critics.

Brownstein, Madva and Kelly examine the role of personal choices in systemic change.

Yudkowsky and Soares argue that we would lose a conflict with artificial superintelligence

Brownstein, Madva and Kelly examine the role of personal choices in systemic change.

Brownstein, Madva and Kelly examine the role of personal choices in systemic change.

This post addresses the second of Yudkowsky and Soares’ two main arguments for misalignment in Chapter 4.

A new way to read the blog.

This post addresses the first of Yudkowsky and Soares' two main arguments for misalignment in Chapter 4.

Yudkowsky and Soares lay out their case for AI risk. This post introduces the book and identifies key cruxes.

Should we expect means-end rational agents to preserve their goals? Southan, Ward and Semler are skeptical.

The second part of the AI 2027 timelines model relies primarily on insufficiently evidenced forecasts.

The AI 2027 report relies on two models of AI timelines. The first timelines model largely bakes hyperbolic growth into the model structure.