iiinova — where your digital self keeps evolving

Margaret Mitchell @mmitchell.bsky.social · 10h

Did you consider the title “attention is REALLY all you need”, hehe.

Margaret Mitchell @mmitchell.bsky.social · 10h

You could see it as: 😏,🐇,🥸,🪶,🥳 is just about as likely as 😜,🐇,🥸,🪶,😵‍💫 Even with identical-looking tokens at the start at the end, it’s still just a sequence of tokens (and perhaps to your paper’s point, even if they look identical at the surface level, they’re not, due in part to their positions)

110

Margaret Mitchell @mmitchell.bsky.social · 11h

Idk, I’ll have to read your paper, but it’s not like even the “same number” on input and output is the same. “101”,”+”,”9”,”=“,”110” being just about as likely as “100”,”+”,”9”,”=“,”109” doesn’t seem unintuitive to me at the form level. It just requires removing my own intuitions about math.😅

200

Margaret Mitchell @mmitchell.bsky.social · 11h

Oh, I see your paper is still asserting the importance of positional information. Yeah that makes sense to me!

120

Margaret Mitchell @mmitchell.bsky.social · 11h

I mean, *you* carry but that doesn’t mean *it* has to carry. Cool work, I’ll check it out!

320

Margaret Mitchell @mmitchell.bsky.social · 11h

Remember these are tokenized with extremely long context windows, with each token associated to a position. So numbers in different positions lead to different outputs. Like, for the number “123456789101”, it’s “1 given ‘1’ 11 positions prior and ‘2’ 10 positions prior and ‘3’ 9 prior” etc

161

Margaret Mitchell @mmitchell.bsky.social · 11h

Yah. I mean it’s not even clearly (standard) addition, esp in all cases (since model-internal math will be different dependent on input). 100+5 has a super high association to 105 without adding 5 items independently. These are operations over forms.

120

Margaret Mitchell @mmitchell.bsky.social · 12h

And then that gets back to what the SP metaphor is actually referring to. /end

040

Margaret Mitchell @mmitchell.bsky.social · 12h

We have evolved to build a mental model of a human mind behind language use, likened to our own lived experiences (this is how we’ve built societies), so it creates this forced imputation of “human” that fundamentally isn’t there. 8/

190

Margaret Mitchell @mmitchell.bsky.social · 12h

But then I admittedly get a little more lost on the discussions of “meaning” and “consciousness”. What I do see is that chat systems are mirroring the kinds of things we write, but without our experiences that give rise to that use of language, which is fundamentally misleading because…7/

151

Margaret Mitchell @mmitchell.bsky.social · 13h

I don’t have a good intuition about whether inference, upon a token input of “+”, could reduce to the simple mathematical operation of addition. Maybe sometimes? 6/

240

Margaret Mitchell @mmitchell.bsky.social · 13h

That relates to solving more complicated math problems too. Associations through multiple layers can connect relevant concepts/derivations. 5/

290

Margaret Mitchell @mmitchell.bsky.social · 13h

The model doesn’t have to be exposed to “5+4+1” to produce “10”. It can take “5+4+1” and after tokenization, the closest neighbor is “9+1”, and “10” is a likely association to that. 4/

1155

Margaret Mitchell @mmitchell.bsky.social · 13h

What I see a bit in your writing (admittedly skimming!) is that you may be treating a Markov chain n-gram model near-identically to an LLM. The “large” part allows for generalizations: 5+4 and 9 are close to each other in vector/embedding space. 3/

190

Margaret Mitchell @mmitchell.bsky.social · 14h

(One thing I’ve learned from operationalizing ethics is that it’s important to pinpoint distinct boundaries so we don’t get lost into meandering convos of talking past one another. So just want to be clear that we’re heading past the realm of the paper itself.) 2/

1120

Margaret Mitchell @mmitchell.bsky.social · 14h

Thanks much. Just to make sure everyone’s on the same page, SP was a paper about the harms and risks of LLMs, drawing largely from previous peer-reviewed work to extrapolate what was coming with LLMs; getting more deeply into other topics generally goes to other papers/different viewpoints. 1/

180

Margaret Mitchell @mmitchell.bsky.social · 02/10/2026

I don't think I understand that question. 😅 We impute meaning into LLM output. That meaning is distinct from what an LLM represents/can be used for generating.

190

Margaret Mitchell @mmitchell.bsky.social · 02/10/2026

Ah, that's where the confusion is: "Doing" addition. In an LLM, "5+4" and "9" would be near synonyms. In the same way "cat" and "feline" are near synonyms. LLMs capture these higher-order relationships (via the "large" part).

230

Margaret Mitchell @mmitchell.bsky.social · 02/10/2026

Also, Emily is correct: Using a calculator makes a lot more sense upon a detection of arithmetic. Most AI systems run classifiers on user input. Good AI system designers, if their goal is a reliable system, would fold in a calculator tool.

3110

Margaret Mitchell @mmitchell.bsky.social · 02/10/2026

Unless I misunderstand something, isn't Colin showing that patterns that are more likely in the training data more reliably captured in model output? That follows from the SP metaphor directly. They're also (implicitly) showing the effects of tokenization. 1/

140

Margaret Mitchell @mmitchell.bsky.social · 02/10/2026

I think Colin (or someone else?) might have blocked me? Can't see the context/evidence. I'm legitimately curious, although time-constrained and can't reliably engage. 🫤

340

Margaret Mitchell @mmitchell.bsky.social · 02/10/2026

I just hope that the actual text behind the concept doesn't get too swept under the rug, b/c it's helpful (gonna have to cut it off here, fam calls...) /end

090

Margaret Mitchell @mmitchell.bsky.social · 02/10/2026

What I see in current discourse is something else, outside of the original paper: Stuff about sentience & consciousness & stuff. I don't write about all that and tend not to opine on it (I feel my opinion isn't relevant). 4/

2120

Margaret Mitchell @mmitchell.bsky.social · 02/10/2026

Unpredictable [stochastic] but human-like patterns [parrot] underlies the "rogue" agent stuff. BUT! 3/

1100

Margaret Mitchell @mmitchell.bsky.social · 02/10/2026

It strikes me that there's a ton going on in current discourse beyond what's in the orig. paper. From my co-author standpoint, a broad-brush simplification is "stochastic"→unpredictable; "parrot"→based on human patterns. That's consistent with where we're at now unfortunately. 2/

1170

Margaret Mitchell @mmitchell.bsky.social · 02/10/2026

As a fan of both you & Emily, I'll just point out something maybe obvious: Searching web, extraction, etc., isn't an LLM. That's engineered code/functions used alongside an LLM; the distinction between an "AI system" and an LLM (I wrote a thing about this here: medium.com/@margarmitch...) 1/

medium.com

No, “AI” is not a Stochastic Parrot 🦜

I’ve recently come across a new flavor of AI denialism making the rounds.

3474

Margaret Mitchell @mmitchell.bsky.social · 15/09/2026

🪶 Captured this great little video of a native Bewick's wren singing to its friends in the forest and it has made my week.

37414

Margaret Mitchell @mmitchell.bsky.social · 15/09/2026

If you read just one thing about the realities of modern AI and agents going "rogue", read this: aiguide.substack.com/p/misleading... Brilliant piece from @melaniemitchell.bsky.social

aiguide.substack.com

Misleading Metaphors, Real Risks

What To Fear from AI Agents and How to Reclaim Our Human Agency

2309

Margaret Mitchell @mmitchell.bsky.social · 05/08/2026

If you are at @deeplearningindaba.bsky.social, remember to check out the Trust AI Workshop!

071

Reposted by Margaret Mitchell

Data & Society @datasociety.bsky.social · 29/07/2026

On August 20, at 1 p.m. ET, the authors will join @mmitchell.bsky.social, chief ethics scientist @hf.co; and Madeleine Claire Elish, head of eval, responsible development, & innovation at Google Deepmind; to discuss their research. Learn more and RSVP! datasociety.net/news-events/...

073

Margaret Mitchell @mmitchell.bsky.social · 29/07/2026

We @hf.co made an interactive visual of the actual hack from the Hugging Face side: the attack chain across trust boundaries, phase activity, and the commands as they were recorded. Key #transparency . huggingface.co/blog/agent-i... Massive props to Hugo, Adrien, Raphael, Christophe for making this.

huggingface.co

Anatomy of a Frontier Lab Agent Intrusion: A Technical Timeline of the July 2026 Incident

We’re on a journey to advance and democratize artificial intelligence through open source and open science.

32811

Margaret Mitchell @mmitchell.bsky.social · 27/07/2026

Fwiw, I'd say a silly cartoon with flying claws to represent data processers etc is distinct from a sober proclamation that an AI system is literally scheming... LLMs are still stochastic and they still parrot; LLMs != AI; And we now wrap them in agentic scaffolds to enable action sequences.

070

Margaret Mitchell @mmitchell.bsky.social · 26/07/2026

8b/8 ... on their own infrastructure, and it worked. The end. Full comic: m-mitchell.com/HF-hack-cart...

m-mitchell.com

The Escape

0221

Margaret Mitchell @mmitchell.bsky.social · 26/07/2026

8a/8 Hugging Face's security systems detected the intrusion. Given the sheer volume of recorded actions, engineers decided to use an AI agent system to analyze what had happened. They first tried a closed system, but its safety guardrails blocked them. So they ran the analysis using an open model...

1151

Margaret Mitchell @mmitchell.bsky.social · 26/07/2026

7/8 Once inside, the agent's commands generated thousands of actions, moving through systems and harvesting credentials.

1111

Margaret Mitchell @mmitchell.bsky.social · 26/07/2026

6/8 Hugging Face processes datasets, and the agent was able to utilize some of the ways they are processed to gain access into Hugging Face internal systems.

2130

Margaret Mitchell @mmitchell.bsky.social · 26/07/2026

5/8 With access to the wider internet, the agent's commands targeted Hugging Face, a major AI platform where ExploitGym answers might be stored.

1120

Margaret Mitchell @mmitchell.bsky.social · 26/07/2026

4/8 With access beyond the sandbox, the agent issued commands that reached through OpenAI's internal research systems, moving from machine to machine until they reached the wider internet.

1130

Margaret Mitchell @mmitchell.bsky.social · 26/07/2026

3/8 The agent uncovered a previously unknown flaw in the proxy that linked to the outside, and exploited it. Instead of just requesting software tools, the agent was able to use the proxy to connect to things outside of the sandbox.

1111

Margaret Mitchell @mmitchell.bsky.social · 26/07/2026

2/8 The sandbox couldn't connect to the internet, but it could request software packages through a "proxy" that would fetch tools from the internet on its behalf and pass them in. This proxy was the sandbox's only link to the outside world.

1120

Margaret Mitchell @mmitchell.bsky.social · 26/07/2026

People not in tech have been interested in what happened with the Open AI agent/Hugging Face hack. So, I "Explain it Like I'm 5"-ed it and made a cartoon in 8 panels. Apologies for some anthro-ing. 🫤 m-mitchell.com/HF-hack-cart... 1/8

An AI agent in development at OpenAI was recently tested on how well it could find and exploit code vulnerabilities.

The test began in a "sandbox", an environment meant to keep everything the agent does contained, so that nothing it triggers can affect systems outside of the test.

613534

Margaret Mitchell @mmitchell.bsky.social · 06/07/2026

If you haven't read this piece on language use in around AI, you definitely should. Great work from @emilymbender.bsky.social , @nannainie.bsky.social and Peter Zukerman, with solutions for what to call things we backoff to just anthropomorphising. firstmonday.org/ojs/index.ph...

Table 3: Suggestions for de-anthropomorphizing language.
Category, Alternative Strategies, Examples.

Cognizer & Products of cognition: Instead of language that locates thinking in an algorithm, describe the algorithm as performing calculations or other algorithmic operations and the people using it doing the thinking. Examples:
artificial intelligence → probabilistic automation

hybrid intelligence → augmented human intelligence

image recognition → image labeling

speech recognition → automatic transcription

the model shows bias → the model reflects bias

model mistakes → model errors

chatbots are good at … → chatbots are good for ….

912644

Margaret Mitchell @mmitchell.bsky.social · 29/06/2026

I am beyond honored to have been selected to give a keynote at Deep Learning Indaba: 4.August, Lagos. I will be cutting into some of the deepest issues I see in AI (hint: it's not alignment) and what I think we can do solve them.

3251

Margaret Mitchell @mmitchell.bsky.social · 24/06/2026

🧊 PSA: if you’re in a heat wave, consider putting ice out in bowls/bird baths for the birds. A lot of them will die otherwise. 😔 🪶

1112

Reposted by Margaret Mitchell

Lauren Dobson-Hughes @ldobsonhughes.bsky.social · 24/06/2026

Musk says not a single person has died due to him axing USAID. Sadly, part of my job involves tracking aid spending. So here’s a thread with just some of the people who lost their lives because of USAID cuts

196133286755