iiinova — where your digital self keeps evolving

Martin Schrimpf @mschrimpf.bsky.social · 04/08/2026

Come say hi at #CCN2026! Scaling laws for visual cortex, topographic multimodal models, video generation for the lateral stream, model-enabled applications, and a GAC on whether #NeuroAI is asking the right questions

0234

Martin Schrimpf @mschrimpf.bsky.social · 07/07/2026

Bonus: the clearest view of Mont Blanc I've ever witnessed, during our lab retreat!

020

Martin Schrimpf @mschrimpf.bsky.social · 07/07/2026

What a treat to host @dyamins.bsky.social at EPFL this past month to dig into the most impactful future of #NeuroAI. A few conclusions we landed on: 1. Task optimization is a powerful starting point for most (all?) brain systems 2. Brain data is the bottleneck 3. Major opportunities in applications

1100

Martin Schrimpf @mschrimpf.bsky.social · 17/02/2026

The core claim you mean is "Models that perform better at predicting the next word in a sequence also better predict brain measurements" -- and yes, that indeed has been validated and extended by many follow-up studies. As you said yourself, the results can also be perfectly replicated.

210

Martin Schrimpf @mschrimpf.bsky.social · 16/02/2026

Regarding the original claims: Badr & others reproduced the correlation to NWP performance with the new benchmarks and newer models so I see no reason for the 2021 claims to be invalid. These new benchmarks are enforcing stronger generalization (great!) but that doesn't mean the old ones were wrong.

210

Martin Schrimpf @mschrimpf.bsky.social · 10/02/2026

2. Splitting across larger temporal chunks (eg stories) is indeed a stronger form of generalization than smaller chunks (eg sentences). @bkhmsi.bsky.social tackled this in his EMNLP'25 where we identified the most stringent evaluation of brain alignment to be linear predictivity with story splits

190

Martin Schrimpf @mschrimpf.bsky.social · 10/02/2026

The perhaps strongest support for this point is that recent LLMs confirm the original prediction: as their task performance improved, their alignment to the human brain further increased (see e.g. Shen et al. 2025).

180

Martin Schrimpf @mschrimpf.bsky.social · 10/02/2026

Thank you Dan for the ping! As far as I can tell, all of the original claims hold, for the following reasons: 1. The relationship between next-word prediction performance and brain alignment has been replicated in several other studies (eg Caucheteux et al 2022; De Varda et al 2025; Mischler 2024).

2120

Martin Schrimpf @mschrimpf.bsky.social · 09/10/2025

To expand on this: When we built our stimulation->neural predictor (www.biorxiv.org/content/10.1...), we didn't find much experimental data to constrain the model. The best we found was data from @markhisted.org and biophysical modeling by Kumaravelu et al.

120

Martin Schrimpf @mschrimpf.bsky.social · 08/10/2025

A glimpse at what #NeuroAI brain models might enable: a topographic vision model predicts stimulation patterns that steer complex object recognition behavior in primates. This could be a key 'software' component for visual prosthetic hardware 🧠🤖🧪

14210

Martin Schrimpf @mschrimpf.bsky.social · 02/10/2025

Digging deeper into the ablated model, we found that its behavioral patterns mirror phonological deficits of dyslexic humans, without a significant deficit in orthographic processing. This connects to experimental work suggesting that phonological and orthographic deficits have distinct origins.

100

Martin Schrimpf @mschrimpf.bsky.social · 02/10/2025

It turns out that the ablation of these units has a very specific effect: it reduced reading performance to dyslexia levels *but* keeps visual reasoning performance intact. This does not happen with random units, so localization is key.

100

Martin Schrimpf @mschrimpf.bsky.social · 02/10/2025

We achieve this via the localization and subsequent ablation of units that are "visual-word-form selective" i.e. are more active for the visual presentation of words over other images. After ablating the units we test the effect on behavior in benchmarks testing reading and other control tasks

100

Martin Schrimpf @mschrimpf.bsky.social · 02/10/2025

I've been arguing that #NeuroAI should model the brain in health *and* in disease -- very excited to share a first step from Melika Honarmand: inducing dyslexia in vision-language-models via targeted perturbations of visual-word-form units (analogous to human VWFA) 🧠🤖🧪 arxiv.org/abs/2509.24597

14912

Martin Schrimpf @mschrimpf.bsky.social · 02/10/2025

110

Martin Schrimpf @mschrimpf.bsky.social · 02/10/2025

100

Martin Schrimpf @mschrimpf.bsky.social · 02/10/2025

100

Martin Schrimpf @mschrimpf.bsky.social · 01/08/2025

which enables a fine-grain mapping of cortical space with a new multi-task relevance analysis; the accurate (R~0.5) prediction of second-by-second human brain activity, which makes us more confident in the characterization of action understanding pathways; and a couple more

110

Martin Schrimpf @mschrimpf.bsky.social · 01/08/2025

The mouse work is definitely relevant, will make sure to reference (apologies for the oversight). I do think there are substantial novelties that have only been made possible with more recent powerful video models: the tight relation to behavior and a variety of tasks,

110

Martin Schrimpf @mschrimpf.bsky.social · 30/07/2025

Where models really shine is their ability to integrate disparate findings. Our findings not only recapitulate known brain structures, they also characterize action understanding pathways. The models help us make sense of hierarchy, behavioral relevance, and functional processing

110

Martin Schrimpf @mschrimpf.bsky.social · 30/07/2025

Brain-like computations support object and motion recognition that map onto classic visual ventral and dorsal streams. But looking deeper, we found a much more distributed computational landscape -- which may emerge from a single computational goal: modeling the visual world

110

Martin Schrimpf @mschrimpf.bsky.social · 30/07/2025

What makes visual processing in the brain so powerful and flexible? Very excited to share our new work where we started from SOTA models that accurately predict dynamic brain activity during hours of video watching, and investigated core computations underlying visual perception

35312

Martin Schrimpf @mschrimpf.bsky.social · 27/06/2025

Have you by any chance tested recent larger models? Would they be more similar? At least for alignment with human classification choices, it seems that scale is really key in closely matching human behavior (plot from arxiv.org/abs/2411.05712)

140

Martin Schrimpf @mschrimpf.bsky.social · 05/05/2025

It depends on how extensive you think "*training* on interventional data" will have to be. Eg, topographic models with a simple "perturbation module" to describe the change in neural activity from interventions were quite powerful in predicting behavioral effects www.biorxiv.org/content/10.1...

2132

Martin Schrimpf @mschrimpf.bsky.social · 12/11/2024

Beyond the distinction of behavioral and neural alignment, there is actually a striking gradient: scaling helps more for higher-level visual regions but only barely for early visual areas.

151

Martin Schrimpf @mschrimpf.bsky.social · 12/11/2024

Across over 600 models that we trained under comparable conditions, we identify increases in *data* size as a stronger driver of alignment over increases in *model* size

120

Martin Schrimpf @mschrimpf.bsky.social · 12/11/2024

This finding is true across different model types, with strong or weak priors (e.g. transformers or conv nets) -- they all converge to the same alignment, with neural alignment saturating

120

Martin Schrimpf @mschrimpf.bsky.social · 12/11/2024

For a while, gains in AI have translated into gains in modeling the brain. We test if that will continue to be the case with recent advances in scaling. Surprisingly we get mixed results: while increased scale advances model alignment to behavior, neural alignment saturates: arxiv.org/abs/2411.05712

13212

Martin Schrimpf @mschrimpf.bsky.social · 12/11/2024

How can we train LLMs with <100M words? In our #BabyLM paper, we introduce a new language+vision self-synthesis training recipe to tackle this question: Our model learns over 4 phases -- most crucially self-captioning unseen images to generate synthetic language data arxiv.org/abs/2411.00828

071

Martin Schrimpf @mschrimpf.bsky.social · 12/11/2024

We actually just rolled out a revamp of the model page so it now more prominently shows the reference. Although this particular submission doesn't seem to have one stored. Based on the name and architecture I'm guessing it's a Convolutional Vision Transformer huggingface.co/docs/transfo...

100

Martin Schrimpf @mschrimpf.bsky.social · 05/11/2024

Applications now open for the Summer@EPFL program summer.epfl.ch -- 3-month fellowship for Bachelor/Master students to immerse yourself in research

010

Martin Schrimpf @mschrimpf.bsky.social · 17/10/2024

Functional responses in the brain to linguistic inputs are spatially organized -- but why? We show that a simple smoothness loss added to language model training explains a range of topographic phenomena in neuroscience: arxiv.org/abs/2410.11516 #NeuroAI #neuroscience #language

0145

Martin Schrimpf @mschrimpf.bsky.social · 25/06/2024

With these findings, we built a simple untrained network "SUMA" with state-of-the-art alignment to brain and behavioral data -- this feature encoder provides representations that are then useful for efficient language modeling 2/

120