iiinova — where your digital self keeps evolving

Read Miranda's full piece here: helentoner.substack.com/p/personaliz... And the research brief she wrote that inspired this post: cdt.org/insights/its...

Personalized AI is rerunning the worst part of social media's playbook

The incentives, risks, and complications of AI that knows you

090

I honestly don't know how big the potential harms of personalization are—I think it's possible we end up coping fine. But it's crazy to me how little mindshare this seems to be getting among people who think about unintended systemic effects of AI for a living.

140

Thinking about this, I keep coming back to two stories— 1) how FB allegedly trained models to identify moments when users felt worthless, then sold that data to advertisers 2) how we're already seeing chatbots addicting kids & adults and warping their sense of what's real

Miranda Bogen @mbogen.bsky.social · 21/07/2025

AI companies are starting to build more and more personalization into their products, but there's a huge personalization-sized hole in conversations about AI safety/trust/impacts. Delighted to feature @mbogen.bsky.social on Rising Tide today, on what's being built and why we should care:

1166

Reposted by Helen Toner

AI companies are starting to promise personalized assistants that “know you.” We’ve seen this playbook before — it didn’t end well. In a guest post for @hlntnr.bsky.social’s Rising Tide, I explore how leading AI labs are rushing toward personalization without learning from social media’s mistakes

open.substack.com

Personalized AI is rerunning the worst part of social media's playbook

The incentives, risks, and complications of AI that knows you

0166

Helen Toner @hlntnr.bsky.social · 30/06/2025

The 3 disagreements are: How far can the current paradigm go? How much can AI improve AI? Will future AI still basically be tools, or will they be something else? Thanks to @farairesearch for the invitation to do this talk! Transcript here: helentoner.substack.com/p/unresolved...

Unresolved debates about the future of AI

How far the current paradigm can go, AI improving AI, and whether thinking of AI as a tool will keep making sense

170

Helen Toner @hlntnr.bsky.social · 30/06/2025

Been thinking recently about how central "AI is just a tool" is to disagreements about the future of AI. Is it? Will it continue to be? Just posted a transcript from a talk where I go into this + a couple other key open qs/disagreements (not p(doom)!). 🔗 below, preview here:

1134

Helen Toner @hlntnr.bsky.social · 16/06/2025

2 weeks left on this open funding call on risks from internal deployments of frontier AI models—submissions are due June 30. Expressions of interest only need to be 1-2 pages, so still time to write one up! Full details: cset.georgetown.edu/wp-content/u...

072

Helen Toner @hlntnr.bsky.social · 19/05/2025

💡Funding opportunity—share with your AI research networks💡 Internal deployments of frontier AI models are an underexplored source of risk. My program at @csetgeorgetown.bsky.social just opened a call for research ideas—EOIs due Jun 30. Full details ➡️ cset.georgetown.edu/wp-content/u... Summary ⬇️

195

Helen Toner @hlntnr.bsky.social · 12/05/2025

...But too many critics of those stasist ideas try to shove the underlying problems under the rug. With this post, I"m trying to help us hold both things at once. Read the full post on AI Frontiers: www.ai-frontiers.org/articles/wer... Or my substack: helentoner.substack.com/p/dynamism-v...

ai-frontiers.org

We’re Arguing About AI Safety Wrong | AI Frontiers

Helen Toner, May 12, 2025 — Dynamism vs. stasis is a clearer lens for criticizing controversial AI safety prescriptions.

132

Helen Toner @hlntnr.bsky.social · 12/05/2025

From The Future and Its Enemies by Virginia Postrel: Dynamism: "a world of constant creation, discovery, and competition" Stasis: "a regulated, engineered world... [that values] stability and control" Too many AI safety policy ideas would push us toward stasis. But...

100

Helen Toner @hlntnr.bsky.social · 12/05/2025

Criticizing the AI safety community as anti-tech or anti-risktaking has always seemed off to me. But there *is* plenty to critique. My latest on Rising Tide (xposted with @aifrontiers.bsky.social!) is on the 1998 book that helped me put it into words. In short: it's about dynamism vs stasis.

143

Helen Toner @hlntnr.bsky.social · 23/04/2025

New on Rising Tide, I break down 2 factors that will play a huge role in how much AI progress we see over the next couple years: verification & generalization. How well these go will determine if AI just gets super good at math & coding vs. mastering many domains. Post excerpts:

162

Stop the World: The road to artificial general intelligence, with Helen Toner

Find them by searching "Stop the World" and "Cognitive Revolution" in your podcast app, or links here: www.aspi.org.au/news/stop-wo... www.cognitiverevolution.ai/helen-toner-...

aspi.org.au

000

Cognitive Revolution (🇺🇸): More insidery chat with @nathanlabenz.bsky.social getting into why nonproliferation is the wrong way to manage AI misuse; AI in military decision support systems, and a bunch of other stuff. Clip on my beef with talk about the "offense-defense" balance in AI:

100

Stop the World (🇦🇺): Fun, wide-ranging conversation with David Wroe of @aspi-org.bsky.social on where we're at with AI, reasoning models, DeepSeek, scaling laws, etc etc. Excerpt on whether we can "just" keep scaling language models:

2 new podcast interviews out in the last couple weeks—one for more of a general audience, one more inside baseball. You can also pick your accent (I'm from Australia and sound that way when I talk to other Aussies, but mostly in professional settings I sound ~American)

120

cc @binarybits.bsky.social re hardening the physical world, @vitalik.ca re d/acc, @howard.fm re power concentration... plus many others I'm forgetting whose takes helped inspire this post. I hope this is a helpful framing for these tough tradeoffs.

010

I don't think this approach will obviously be enough, I don't think it's equivalent to "just open source everything YOLO," and I don't think any of my argument applies to tracking/managing the frontier or loss of control risks. More in the full piece: helentoner.substack.com/p/nonprolife...

Nonproliferation is the wrong approach to AI misuse

Making the most of “adaptation buffers” is a more realistic and less authoritarian strategy

100

What to do instead? IMO the best option is to think in terms of "adaptation buffers," the gap between when we know a new misusable capability is coming and when it's actually widespread. During that time, we need massive efforts to build as much societal resilience as we can.

120

The basic problem is that the kind of AI that's relevant here (for "misuse" risks) is going to get way cheaper & more accessible over time. This means that to indefinitely prevent/control its spread, your nonprolif regime will get more & more invasive and less & less effective.

Helen Toner @hlntnr.bsky.social · 03/04/2025

Seems likely that at some point AI will make it much easier to hack critical infrastructure, create bioweapons, etc etc. Many argue that if so, a hardcore nonproliferation strategy is our only option. Rising Tide launch week post 3/3 is on why I disagree 🧵 helentoner.substack.com/p/nonprolife...

4174

The idea of AI "alignment" seems increasingly confused—is it about content moderation or controlling superintelligence? Is it basically solved or wide open? Rising Tide #2 is about how we got here, and how the core problem is whether we can steer advanced AI at all.

The core challenge of AI alignment is “steerability”

"Steered to where" is a different question from "steerable at all"

050

Helen Toner @hlntnr.bsky.social · 01/04/2025

Sharing the very first post on my new substack, about the weird boiling frog of AI timelines. Somehow expecting human-level systems in the 2030s is now a conservative take? 2 more posts to come this week, then a slower pace. Subscribe, tell your friends! helentoner.substack.com/p/long-timel...

"Long" timelines to advanced AI have gotten crazy short

The prospect of reaching human-level AI in the 2030s should be jarring

091

Secure Enclaves for AI Evaluation - OpenMined

More technical detail from OpenMined on the secure enclaves on H100s they used to make the Anthropic/UKAISI pilot happen: openmined.org/blog/secure-... And a call for proposals on a related topic from Open Philanthropy: openphilanthropy.org/request-for-...

openmined.org

Andrew Trask*, Aziz Berkay Yesilyurt*, Bennett Farkas*, Callis Ezenwaka*, Carmen Popa*, Dave Buckley*, Eelco van der Wel*, Francesco Mosconi‡, Grace Han‡, Ionesio Junior*, Irina Bejan*, Ishan Mishra§,...

010

Still early days for this tech but it's cool stuff, and we're proud that @csetgeorgetown.bsky.social's Foundational Research Grants program helped to fund OpenMined's work on this.

Pilot 2, even cooler: Anthropic & UK AISI did an exercise in "mutual secrecy," using OpenMined's tech to run a biosecurity evaluation where the AI model was kept private from the UK govt and the bio dataset was private from Anthropic. Just a test exercise so far, but v promising.

In pilot 1 DailyMotion, a French video sharing site, connected OpenMined's privacy-preserving infrastructure to their stack. This let an external researcher analyze what kind of content they were upranking *without* needing access to user data or the algorithm 🤯

Enabling External Scrutiny of AI Systems with Privacy-Enhancing Technologies

New short paper from Kendrea Beers and me - 2 case studies of @openmined.bsky.social's great work giving researchers/auditors/etc access to test privately held AI models. Thread with more on the two successful pilots below. Paper is here: arxiv.org/abs/2502.05219

arxiv.org

This article describes how technical infrastructure developed by the nonprofit OpenMined enables external scrutiny of AI systems without compromising sensitive information. Independent external scru...

151

Helen Toner @hlntnr.bsky.social · 12/03/2025

Lately it sometimes feels like there are only 2 AI futures on the table—insanely fast progress or total stagnation. Talked with @axios.com's Alison Snyder at SXSW about the many in-between worlds, and all the things we can be doing now to help things go better in those worlds.

1112

Helen Toner @hlntnr.bsky.social · 05/03/2025

New podcast ep! Had fun talking US-China competition, military AI, and more with @spencrgreenberg.bsky.social: podcast.clearerthinking.org/episode/251/... Even spent a few min at the end on something I rarely get to ramble about: what I've learned about relating to humans from working with horses🐴

podcast.clearerthinking.org

092

Helen Toner @hlntnr.bsky.social · 13/01/2025

Due in 2 days (!): comments on v2 of the EU's draft Code of Practice for general-purpose AI. For anyone else who'd find it helpful, here's a side-by-side comparison with v1: draftable.com/compare/NNBy... One snap take: the rewritten taxonomy of systemic risks (p29) is much stronger.

030

Oh awesome, will read! I've been wanting to write more reflections on/lessons from the book, but wanted to learn more about nuclear regulation in the 80s and 90s first. Thanks for finding & sharing this.

020

Lastly, a book on nuclear industry self-regulation to improve safety post Three Mile Island. Far from a how-to guide for AI, not least because nuclear energy basically died in the US under this regime. But many interesting (dis)analogies to ponder. www.amazon.com/Hostages-Eac...

amazon.com

Positively heartwarming NYT story on the Little Taipei springing up near TSMC's new plant in Arizona. Beef noodle soup, English lessons at church, high-skill immigrant visas... everything a progress studies nerd's heart could desire. www.nytimes.com/2024/12/29/b...

150

Fun @jascha.sohldickstein.com post that straightforwardly connects 2 ways of thinking about the perils of optimization (Goodhart's Law and overfitting), then suggests some ways to apply anti-overfitting measures to societal problems. Thought-provoking! sohl-dickstein.github.io/2022/11/06/s...

291

By Anton Leicht: how well did AI safety advocates make use of the post-ChatGPT policy window? Some good nuance re: how hard it is to regulate tech that doesn't exist yet, and where the AI safety crowd made mistakes vs. incurred hard-to-avoid costs. www.techpolicy.press/the-end-of-t...

140