Whether or not it will bring about our extinction, AI clearly risks causing many kinds of catastrophic harms. Why are we giving ruthless private companies such power to shape the human future?