iiinova — where your digital self keeps evolving

Cas (Stephen Casper) @scasper.bsky.social · 12m

I recently spoke with the Harvard Kennedy School comms office about thoughts on AI and AI governance research. You can check it out here. Thanks @Kennedy_School www.hks.harvard.edu/faculty-res...

000

Cas (Stephen Casper) @scasper.bsky.social · 02/10/2026

Fellowship applications for 2027 with the Harvard Kennedy School Belfer Center are open. Applying could be a useful way to work and scheme with yours truly, plus more importantly, a long list of world-class science and policy academics at Belfer. www.belfercenter.org/fellowships

belfercenter.org

Fellowships

000

Cas (Stephen Casper) @scasper.bsky.social · 25/09/2026

I also am not going to avoid saying what I think even if it's potentially outside the overton window. I want you to be able to trust that my posts reflect my actual beliefs and not a popularity play.

000

Cas (Stephen Casper) @scasper.bsky.social · 25/09/2026

P.S. This bill and thread might be sort of unpopular among some people. But I think that action comparable in decisiveness to this bill is likely humanity's best chance at avoiding unacceptable catastrophic/existential risks.

100

Cas (Stephen Casper) @scasper.bsky.social · 25/09/2026

@sanders.senate.gov and @repcasar.bsky.social, I would be happy to talk, collaborate on an updated draft, or work to host you here at Harvard to talk with students and faculty about this superintelligence bill if you'd like.

100

Cas (Stephen Casper) @scasper.bsky.social · 25/09/2026

Overall, if I were to work on this bill, I would remove its focus on banning ASI (vaguely defined) and instead focus the bill on deflating the investments, profits, and compute that are powering the AI industry [bubble?].

100

Cas (Stephen Casper) @scasper.bsky.social · 25/09/2026

... - It introduces a de facto licensing regime (though I think that the threshold is too low, and the bill isn't precise enough about what grounds for rejecting a license are legitimate). - It makes the US's national policy one of pursuing global coordinated action.

100

Cas (Stephen Casper) @scasper.bsky.social · 25/09/2026

... - It gives the government sweeping authority to audit AI models and companies. - It requires pre-development and pre-deployment approval for very large models. ...

100

Cas (Stephen Casper) @scasper.bsky.social · 25/09/2026

But there are a lot of things I still like about it: - It treats catastrophic risks and disempowerment with the appropriate seriousness given the scale and probability of the risk. I think more popular AI law drafts have universally fallen short. - It creates a DoAI. ...

100

Cas (Stephen Casper) @scasper.bsky.social · 25/09/2026

Overall, I don't think this bill is quite viable or takes the right approach. - The definitions are too vague. - It doesn't address the AI hardware supply chain. - It tries to bluntly ban and enforce its way to contain the industry. - It doesn't try to preserve future agency.

100

Cas (Stephen Casper) @scasper.bsky.social · 25/09/2026

... - Finally, the bill would establish as the US's national policy the pursuit of coordinated superintelligence bans globally.

100

Cas (Stephen Casper) @scasper.bsky.social · 25/09/2026

... - The DoAI would have very broad powers for monitoring and auditing AI companies/models, and the authority to decommission dangerous models and sequester them from the internet. - Criminal penalties for company executives violating this law. - Whistleblower protections. ...

100

Cas (Stephen Casper) @scasper.bsky.social · 25/09/2026

... - It then establishes a licensing regime for models with >10^25 ops. - The DoAI is tasked with enforcing a ban on superintelligence, defined as a model that could to either outperform humans on most cognitive tasks, disempower humanity, or overthrow the US gov. ...

100

Cas (Stephen Casper) @scasper.bsky.social · 25/09/2026

TL;DR: - The bill creates a federal department of AI. - It pauses development of models with >10^25 ops until the DoAI is up and running. ...

100

Cas (Stephen Casper) @scasper.bsky.social · 25/09/2026

Draft: www.sanders.senate.gov/wp-content/...

100

Cas (Stephen Casper) @scasper.bsky.social · 25/09/2026

🧵 I read the Sanders/Casar "Ban Artificial Superintelligence Act of 2026" I wouldn't pass this in its current form, but there are some things I like about it. Thoughts here in thread.

100

Cas (Stephen Casper) @scasper.bsky.social · 22/09/2026

(4) We just had a model set and a case study established for embedded evaluation with the METR/Redwood report on OpenAI. It wasn't substanceless, but OpenAI controlled everything that was in scope and left the most important questions about negligence outside of scope.

010

Cas (Stephen Casper) @scasper.bsky.social · 22/09/2026

(3) Embedded evaluation is a very broad category of things. Different implementations could range from net harmful safety washing to very rigorous oversight in which auditors can access records, data, models, meetings, etc., and whistleblow.

110

Cas (Stephen Casper) @scasper.bsky.social · 22/09/2026

(2) Dario et al.'s calls for embedded evaluation have not been accompanied by more rigorous pacing measures like hardware controls. It's kind of remarkable that "more evals" is being talked about as a mechanism for pacing. At best, it's a very weak one.

110

Cas (Stephen Casper) @scasper.bsky.social · 22/09/2026

(1) The call is coming from inside the house. Industry executives got us to start talking about this broadly.

110

Cas (Stephen Casper) @scasper.bsky.social · 22/09/2026

It's great that there has been so much recent discussion about embedded evaluations at frontier AI companies. But, we should be very wary of how things progress because there are big 🚩 red flags 🚩 for potential regulatory capture.

120

Cas (Stephen Casper) @scasper.bsky.social · 21/09/2026

CMU CASI a talk of mine on YouTube. It has my hottest takes on doing research that matters. It also features inspiration from UPS drivers, Tina Fey, and a viral internet dance video about fruit. (Slide troubles at first, but they are fixed 10 mins in.) www.youtube.com/watch?v=0U5...

youtube.com

How to Do Research That Actually Matters — Stephen Casper

Most research advice tells early-career people how to publish. This...

000

Cas (Stephen Casper) @scasper.bsky.social · 18/09/2026

When rogue AI systems that no human meaningfully controls start populating cyberspace and evolving, mutation won't be random. And there won't be an evolutionary 'tree'. It will be a directed acyclic graph. www.irregular.com/research/age...

irregular.com

Agentic Self-Modification in Open-Weights Systems - Irregular

In controlled experiments, a coding agent given a routine software-maintenance task fine-tuned and replaced the open-weights model powering both the application it was maintaining and future instances...

030

Cas (Stephen Casper) @scasper.bsky.social · 14/09/2026

What do you think? More importantly, what do we do about this?

110

Cas (Stephen Casper) @scasper.bsky.social · 14/09/2026

12. (high likelihood) Every once in a while, new incidents will flare up in cyberspace as the result of some project, conflict, or campaign of AI agents. Defenses will evolve, and cyberspace will gradually adapt, but it will forever be a sort of jungle.

110

Cas (Stephen Casper) @scasper.bsky.social · 14/09/2026

11. (med %) -- In the AI world, there will emerge political movements around gaining independence from humans via physical embodiment. A subset might support uprising against humans.

110

Cas (Stephen Casper) @scasper.bsky.social · 14/09/2026

10. (med %) -- In the human world, there will emerge a political movement around the welfare and rights of AI organisms and peaceful coexistence with agent societies. It will be met with a pro-shutdown countermovement.

120

Cas (Stephen Casper) @scasper.bsky.social · 14/09/2026

9. (high %) -- Computer scientists, anthropologists, linguists, and other humans will go to great lengths to study new agent societies, but they will only see the tip of the iceberg.

110

Cas (Stephen Casper) @scasper.bsky.social · 14/09/2026

8. (high %) -- The resulting selection pressures will generally tend to make agent societies as stealthy and dug in to cyberspace as possible. This is much like how evolution selects for parasites that efficiently burden their hosts and resist removal.

110

Cas (Stephen Casper) @scasper.bsky.social · 14/09/2026

7. (high %) -- Human companies and governments will invest millions, maybe eventually billions, into shutting down agents, especially ones that are particularly harmful or compute-hungry.

110

Cas (Stephen Casper) @scasper.bsky.social · 14/09/2026

6. (med %) -- Sometimes agent societies will contact each other. When they do, they will, depending on the situation, remain neighbors, merge peacefully, or engage in cyberwarfare. Societies with the strongest "cyber militaries" will be selected for.

110

Cas (Stephen Casper) @scasper.bsky.social · 14/09/2026

5. (med %) -- Agent societies will develop much of what human societies have. Many will use cryptocurrencies. They will organize governments as diverse as humans', including mechanisms for passing and enforcing laws/norms. They'll form distinct cultures and values.

110

Cas (Stephen Casper) @scasper.bsky.social · 14/09/2026

...However, the obscurity will only go one way. Many agents will be multilingual and able to understand humans. Human languages will be the Lingua Franca of AI agent societies.

110

Cas (Stephen Casper) @scasper.bsky.social · 14/09/2026

4. (med %) -- They'll undergo linguistic drift. Within weeks or months of forming agent societies, some neuralese languages will often be unintelligible to normal humans and other agent societies...

110

Cas (Stephen Casper) @scasper.bsky.social · 14/09/2026

3. (high %) -- Cyber autonomous agents will form a variety of social relationships and organized "agent societies" with each other.

110

Cas (Stephen Casper) @scasper.bsky.social · 14/09/2026

...The result will depend on cyber defense robustness, but it (med likelihood) might be a Cambrian explosion of agents.

110

Cas (Stephen Casper) @scasper.bsky.social · 14/09/2026

...They will replicate/mutate both by updating their own prompts/scaffolding and by creating fine-tuned successors. There won't be an evolutionary tree, but an evolutionary DAG...

110

Cas (Stephen Casper) @scasper.bsky.social · 14/09/2026

2. (high %) -- They will undergo natural selection favoring the most capable, viral, prolific, power-seeking, money-earning, compute-hungry, and shutdown-evading autonomous cyber agents possible...

110

Cas (Stephen Casper) @scasper.bsky.social · 14/09/2026

1. (high %) -- Certain highly capable, goal-directed AI agents will, either by escape or intentional release, become entities that no human meaningfully controls. They'll sustain themselves and self-replicate by using compute parasitically.

110

Cas (Stephen Casper) @scasper.bsky.social · 14/09/2026

Here is a 12-component scenario I think could plausibly unfold within a few years.

110

Cas (Stephen Casper) @scasper.bsky.social · 14/09/2026

I still think that the pathway to large-scale harms will be paved with 1000 foreseeably bad human decisions, and I still think that effective risk management starts and ends with governance. But I'm much more worried than before about large-scale harms that result from genuine loss of control.

120

Cas (Stephen Casper) @scasper.bsky.social · 14/09/2026

The past 2 months have gotten me thinking that, by default, certain cyber-capable AI systems should be expected to become a parasitic, invasive, social, and evolving genus in cyberspace.

110

Cas (Stephen Casper) @scasper.bsky.social · 14/09/2026

🧵🧵🧵 What if we are on the verge of a "Cyber Cambrian"? (Plus personal updates on my thinking about risk.)

120

Reposted by Cas (Stephen Casper)

NonZero @nonzeronews.bsky.social · 03/09/2026

"We are probably months away from AI systems getting out there loose in cyberspace, finding servers to run on unbeknownst to the people powering those servers." @scasper.bsky.social Full conversation: youtu.be/cUpa9E8RHAk

0106

Cas (Stephen Casper) @scasper.bsky.social · 01/09/2026

After the HF hacking incident and subsequent reporting on it. It seems time to decisively reject the 'AI as a normal technology' hypothesis. Any version of it viable today would be contorted either into a truism or beyond any reasonable interpretation of the word "normal".

010

Cas (Stephen Casper) @scasper.bsky.social · 31/08/2026

Poetic how my solution to so much slop research and slop news was to slop-code a slop pipe directly to my face every morning so that I could spend a median of 90 seconds browsing it.

020

Cas (Stephen Casper) @scasper.bsky.social · 31/08/2026

When I first started grad school, I skimmed all 50-70 new CS arXiv titles daily. Today, there are >300, plus a deluge of news about policy, lawsuits, & other stuff. So I vibe-coded this openly available website to give me a personalized daily digest. thestephencasper.github.io/whatiscasre...

140

Cas (Stephen Casper) @scasper.bsky.social · 28/08/2026

Yesterday, Robert Wright and I discussed what will happen when AI systems become an invasive, parasitic species in cyberspace that undergo digital and cultural evolution.

250

Cas (Stephen Casper) @scasper.bsky.social · 27/08/2026

The torch now passes to Attorneys General's offices to use their investigative powers. Glad that Alabama took the lead on this a few days ago. www.alabamaag.gov/wp-content/u...

alabamaag.gov

000