iiinova — where your digital self keeps evolving

pengrui-han.bsky.social @pengrui-han.bsky.social · 21/09/2026

So excited and honored to be selected as a Siebel Scholar, Class of 2027! I’m incredibly grateful to my advisor Jiaxuan You and to all my mentors and collaborators for their support and encouragement along the way! www.businesswire.com/news/home/20...

000

Reposted by pengrui-han.bsky.social

Andrea de Varda @andreadevarda.bsky.social · 01/07/2026

Like the human brain, LLMs use separate sets of units for language, formal reasoning, social reasoning, and intuitive physical reasoning. A modular organization of cognition may be a fundamental principle of intelligence!

0133

Reposted by pengrui-han.bsky.social

Ev Fedorenko @evfedorenko.bsky.social · 30/06/2026

I am so excited about this finding from @pengrui-han.bsky.social and @andreadevarda.bsky.social, also with Jacob Andreas! Perhaps modularity is inevitable in intelligent systems, biological or in silico. :)

0336

pengrui-han.bsky.social @pengrui-han.bsky.social · 30/06/2026

11/n This also opens new directions: deriving testable fMRI predictions about which network a new task engages, studying how information flows between networks (hard to do in the brain), and asking whether the modularity built into AI like mixture-of-experts brings advantages beyond efficiency.

050

pengrui-han.bsky.social @pengrui-han.bsky.social · 30/06/2026

10/n A second kind of intelligent system lets us ask not just whether modularity arises, but why. One idea: brain modularity minimizes metabolic cost. But LLMs face no such cost, so that can't be the driver. Modularity may instead keep cognitive operations from interfering when they share an input.

010

pengrui-han.bsky.social @pengrui-han.bsky.social · 30/06/2026

10/n These neurons also have domain-specific causal effects: lesioning the neurons identified for one domain selectively impairs that same domain over others, a ten-fold difference on average. The dissociation is striking and selective.

010

pengrui-han.bsky.social @pengrui-han.bsky.social · 30/06/2026

9/n We find LLMs develop modular organization resembling the human brain: tasks on the same network share neurons, tasks on different networks don't. Within-domain overlap is 4x cross-domain, clustering recovers the four neuroscience domains, and the structure holds across 6 LLMs (24–123B params).

070

pengrui-han.bsky.social @pengrui-han.bsky.social · 30/06/2026

8/n (3) Score each neuron by combining how much its activation shifts between the two inputs with how much the output depends on it, approximating each unit's causal contribution. (4) Measure how much these units overlap across tasks and domains.

020

pengrui-han.bsky.social @pengrui-han.bsky.social · 30/06/2026

7/n To find the units supporting these tasks, we use attribution patching, in four steps. (1) Start from each task's minimal pairs. (2) Pass both inputs through the model and record activations at every neuron.

010

pengrui-han.bsky.social @pengrui-han.bsky.social · 30/06/2026

6/n Each task is built from minimal pairs. An "original" and an "alternative" input differ in a single task-relevant detail (e.g., 86 + 15 vs 86 + 16) but flip the correct answer (101 vs 102). This isolates the specific computation each task depends on.

010

pengrui-han.bsky.social @pengrui-han.bsky.social · 30/06/2026

5/n We tested 46 tasks across four domains, grounded in brain networks studied in humans: language (supported by the language network), formal reasoning (Multiple-Demand network), physical reasoning (Intuitive Physics network), and social reasoning (Theory of Mind network).

041

pengrui-han.bsky.social @pengrui-han.bsky.social · 30/06/2026

4/n This project has been in the works for a while :) Huge thanks to my advisors Jacob Andreas @evfedorenko.bsky.social @andreadevarda.bsky.social , and to @nancykanwisher.bsky.social for valuable conceptual input and feedback throughout. #MIT

030

pengrui-han.bsky.social @pengrui-han.bsky.social · 30/06/2026

3/n Using circuit analyses across 46 tasks spanning four cognitive domains, we find: 1️⃣ Tasks that draw on the same network in humans recruit overlapping units in LLMs, while tasks drawing on different networks recruit distinct units. 2️⃣ These units are causally linked to model behavior.

030

pengrui-han.bsky.social @pengrui-han.bsky.social · 30/06/2026

The human brain is strikingly modular: distinct networks for language, formal reasoning, social reasoning, physical reasoning. Is this fundamental to intelligent systems, or an accident of evolution? In our new preprint, we find the same modular organization emerges in LLMs.

129320