iiinova — where your digital self keeps evolving

Agreed they're a nice controlled case to study!

000

But I think that's a "the science is far from complete because we just started studying these systems" issue, not an "it's impossible to do science on generalization in these systems" issue. 4/4

150

Now, I do think for things like robustness or mathematical creativity you *do* need sufficient training diversity, which as you say requires frontier models, and makes it harder to make as concrete of links between them and smaller experiments just yet. (Obv. lab secrecy makes it harder.) 3/4

120

On the generalization of language models from in-context learning and finetuning: a controlled study

and/or by reducing those elements down to simpler small settings where everything is controlled. E.g., some of our work taking those two paths to studying relational generalization (arxiv.org/abs/2505.00661 and arxiv.org/abs/2509.16189) or causal reasoning (proceedings.neurips.cc/paper_files/...) 2/

arxiv.org

Large language models exhibit exciting capabilities, yet can show surprisingly narrow generalization from finetuning. E.g. they can fail to generalize to simple reversals of relations they are trained...

I agree with most of what you said, and especially that these are good questions to ask! I guess a core premise of (part of) my research program has been that we *can* study these issues by things like creating controlled, novel tests on which to evaluate generalization of large models 1/4

130

Indeed, I guess everyone probably has the ick response at some form of reference to AI. But it doesn't seem ideal for our discussions to revolve around people's emotional reactions (though I'm also not sure how to avoid it)

020

I do think that people have an ability to continue improving in an area of their expertise in a way that current models cannot as effectively, but I'd argue what they do is closer to moving the domain in distribution by working on it than generalizing OOD zero-shot.... 4/4

130

And similarly the people who prove something like the kinds of math results that LMs have recently been proving tend to have worked in that area for a long time. 3/

110

Now, I think that students who pursued math tended to get better at generalizing within math, but my sense is that is primarily by *studying* it and seeing the repeated structures and strategies (indeed some classes explicitly teach that) 2/

110

I would agree that AIs performance drops OOD. But these arguments seem to often presuppose that human performance doesn't, and I think that's just clearly wrong. I used to TA calc (at a good school) and it was pretty clear that people strongly over it to the problems they saw in class / the book 1/

150

I agree that they are different learning processes and efficiencies. I am not sure why that should change questions of generalization, which we can already demonstrate at small scales. (And of course, it's worth noting that we train models to be competent across many more domains than any person.)

I argue that describing LMs as having goals, beliefs etc. clearly meets that bar. Even if the metaphors are not always perfect, that doesn't mean we have to discard them entirely. Instead, we can use them in ways that acknowledge their imperfections. infinitefaculty.substack.com/p/in-defense... 2/2

3120

New blog post! I wrote about why I think it's useful to use metaphorical (and anthropomorphic) terms to describe AI. I start by describing the role of metaphor in language and cognition, and how metaphors can help us to efficiently understand relational structure and make predictions 1/2

In defense of metaphor: why metaphorical descriptions of AI are useful, even if they are sometimes imperfect

A watercolor of an MLP, with a thought bubble coming out containing a path to a goal

2476

We spend a long time educating scientists in how to reason about science, even though we think humans can generalize. Generalizing is not easy.

130

Well surely if you don't think these models can do any interesting generalization out of distribution, data doesn't matter? Since all the interesting tasks in science will by definition always be out of distribution :)

100

Ben Hayden @benhayden.bsky.social · 02/10/2026

Are you: * A cognitive science (or adjacent) PhD (current or graduated, or equivalent research exp.)? * Interested in studying LMs abilities for conceptual reasoning about cog sci, and helping to improve them? If so, apply to our new conceptual reasoning fellows program! Link below:

1277

Reposted by Andrew Lampinen

New paper from the Neurosurgery research team at BCM! 🧵 "A population code for semantics in human hippocampus" led by Melissa Franch. We recorded hippocampus single neurons while people listened to podcasts. www.nature.com/articles/s41...

nature.com

A population code for semantics in human hippocampus - Nature Neuroscience

Franch et al. show that human hippocampal neurons encode the meanings of the words we hear through distributed, context-sensitive population activity that mirrors some features of large language model...

110132

And for content effects on reasoning: academic.oup.com/pnasnexus/ar...

020

Thanks to the organizers (including @robmok.bsky.social) for inviting me to this fun conference! For more on the work, some older threads: Latent learning bsky.app/profile/lamp... And hippocampal contributions to generalization as data augmentation: bsky.app/profile/lamp...

150

I illustrated this idea with two case studies from our work where we've found what seem to be hints of some shared principles: content effects on logical reasoning, and latent learning / replay as data augmentation. 4/

Two case studies where we see (hints of) shared principles between natural and artificial intelligence:

Drawing of arobot and a person sharing a thought bubble containing formal logic. Below, the heading reads "Content effects on reasoning: a rational analysis," followed by bullet points stating that language models, like humans, mix content into their logical reasoning, and that this may be rational (most accurate in expectation) for an imperfect reasoner.
• Right column: An illustration of a human brain with a highlighted hippocampus connects to a thought bubble showing a graph with nodes X, Y, and Z. Solid blue arrows link X to Y and Y to Z, with a dashed red arrow directly from X to Z. Below, the heading reads "Latent learning & replay as data augmentation," with bullet points explaining that learning experiences latently convey more than explicit content, and that episodic storage allows flexible use of these experiences both offline (augmentation/preplay) and online (recall).

We can then test that invariance further on the AI side, e.g. by seeing how changing the structure of data, or the system's constraints, change the solution. 3/

But I suggested that these differences could actually be useful — where the systems nevertheless arrive at similar solutions, that points toward what invariant computational properties solutions to that task might need to have. 2/

if two systems with very different affordances and histories arrive at the same solution, perhaps the solution space is actually constrained
(illustration of the robot and monkey taking different paths to the same goal)
maybe these differences will actually make it easier to find the *invariant* computational principles of intelligence?

180

Andrew Lampinen @lampinen.bsky.social · 27/09/2026

I gave a talk yesterday at a CiNeT / University of Osaka conference about AI & neuroscience. I started with this question — given that AI (especially language models) and brains are so different in structure and environment, how could they have anything to do with one another? 1/

how could AI and neuroscience have anything to do with each other? (very different illustrations of a robot in a world of binary, and a monkey in a rainforest)

1273

But conditioning it on multimodality is still evidence of getting the basics wrong. I'm aware of almost no emotional evidence that multimodal learning improves anything that language models do in language alone, and many of the controlled settings I'm thinking of don't require it for capabilities.

141

Which is a shame, because I actually think there are such deep and fascinating issues to be understood about what it means for a system to have understanding, to reason, etc. by engaging with what these systems do (and do not) accomplish! 10/10

0170

But the thing that frustrates me is that the authors never engage with empirical evidence they disagree with. It's fine to make a poor argument with flashy rhetoric! But you should consider updating as the evidence changes. And the authors clearly don't. 9/

1161

infinitefaculty.substack.com

critiquing arguments about it being impossible to learn meaning from from as "proving too much" x.com/AndrewLampin... and discussing the relationship between memorization and generalization in LMs and other ML systems: infinitefaculty.substack.com/p/memorizati... 7/

Memorization vs. generalization in deep learning: implicit biases, benign overfitting, and more

Or: how I learned to stop worrying and love the memorization

190

Passive learning of active causal strategies in agents and language models | Proceedings of the 37th International Conference on Neural Information Processing Systems

I've spent a lot of time working on issues in this space, e.g. about whether language models could learn about causal structures and reasoning from passive offline training (dl.acm.org/doi/10.5555/...) 6/

dl.acm.org

190

It's convenient that the authors never really define any of their terms or make falsifiable claims; it's therefore still possible to fight a rearguard action about how it's just harnesses that magically provide meaning — despite tool calls just being tokens like any other language. 5/

1140

It's that argument that modern LMs (and our scientific understanding of their learning and generalization) disprove. The kind of generalizations that language models trained from scratch in controlled settings exhibit (without requiring tools, harnesses, etc.) are sufficient. 4/

170

This argument is unchanged by post-training, RL, etc. rather than simply language modeling (indeed, Bender & Koller talk about systems post-trained for linguistics tasks) — and thus, as others note, the authors still maintain it applies to modern LLMs 3/

280

The core technical claim (drawn from Bender & Koller) of the stochastic parrots paper about LMs is that systems trained on language alone can only learn its form, not its meaning (which requires grounding). Thus, that *a priori* these systems could not exhibit understanding. 2/

181

Andrew Lampinen @lampinen.bsky.social · 23/09/2026

The phrase "stochastic parrot" is full of sound and fury, signifying nothing. Which, ironically, is exactly what the authors got wrong about language, and the core technical mistake of the paper. 1/ (cross-quote-post because I think this topic is important)

Melanie Mitchell on Twitter
Everyone! This is a straw-person argument. The Stochastic Parrot paper was about LLMs of 2021, not the AI of today, which are not LLMs but complex software systems with vast post training and many external software components.

5553

Congrats! Very well deserved :)

010

Reposted by Andrew Lampinen

Tomer Ullman @tomerullman.bsky.social · 22/09/2026

new preprint: "Directing large language models to follow the letter or spirit of the law" arxiv.org/pdf/2609.23083 (by Qian , Li, Chen, Murthy @soniakmurthy.bsky.social , Belinkov, and me) this is particularly cool/important, and I'm allowed to say it because it was headed by @pqian.bsky.social

17423

Andrew Lampinen @lampinen.bsky.social · 21/09/2026

New post reflecting on recent AI progress in math, how AI is changing the way we work, and some worries about where people will find meaning as they offload more of their work to AI: infinitefaculty.substack.com/p/math-resea...

infinitefaculty.substack.com

Math, research, and meaning in the age of AI

Since I wrote my last post about AI and math, the theorems have continued to fall.

0372

Reposted by Andrew Lampinen

Jane Li 🦖 @janeli.bsky.social · 17/09/2026

🦀New preprint! (w/ @najoung.bsky.social)🦞 Is grammaticality a major organizing principle of NLM representations? We show that many NLMs exhibit abstract rep. separation for grammaticality. We believe this work addresses debates about confounds in measuring model gram. knowledge. [1/10]

12010

Reposted by Andrew Lampinen

Timothy Gowers @wtgowers.bsky.social · 17/09/2026

I've written a blog post responding to the letter about maths and AI signed by 25 Fields medallists. As with the Leiden Declaration, I didn't sign it, but I agree with much of it and welcome its existence. gowers.wordpress.com/2026/09/17/w...

gowers.wordpress.com

Why I didn’t sign the Fields medallists’ letter

When I was around 11 I heard for the first time about Fermat’s Last Theorem. I was immediately captivated by the problem statement, as well as by the accompanying story, and made a fairly ser…

45523

Reposted by Andrew Lampinen

Sam Nastase @samnastase.bsky.social · 18/08/2026

New perspective piece out in @cp-neuron.bsky.social with Zaid Zada, @adelegoldberg.bsky.social, and Uri Hasson! We try to articulate some of our excitement about LLMs and discuss what kinds of insights they might provide into the neural computations supporting natural language in the human brain.

18725

Andrew Lampinen @lampinen.bsky.social · 15/08/2026

The brain learning that could be (/probably is, at least in part) over evolutionary time of course

000

And you can find some of our past work that inspired us to think about these connections here: bsky.app/profile/lamp...

010

Check out the paper if you're interested! kwnsfk27.r.eu-west-1.awstrack.me/L0/https:%2F... (or the preprint: arxiv.org/abs/2608.01297). Thanks to @tylerbonnen.bsky.social for the fun collaboration! 4/

120

And we argue that these ML tools can help to build linking functions that help to more concretely link theories of hippocampal function to the real neural computations and behaviors that they are meant to explain. 3/

110

Generalizing from limited data is hard. Hippocampal-lesions lead to selective generalization deficits. We argue that these deficits parallel generalization failures in ML that data augmentation helps address — and augmentation strategies are reflected in hippocampal replay. 2/

An overview of our perspective, relating machine learning findings and methods to phenomena of hippocampal contributions to generalization. (top)
The deficit: machine learning systems without data augmentation, and animals with hippocampal lesions, show impaired generalization. (middle) Data
augmentation has been shown to ameliorate these deficits in machine learning, and we argue that the hippocampus may contribute to a similar
process. (middle, second row) Data augmentation can take various forms, including refactoring an individual learning experience based on prior
knowledge or recombining multiple experiences. (bottom) Ultimately, we aim for an end-to-end, stimulus computable model that predicts how neural
activity and behavior depend on individual learning experiences

220