iiinova — where your digital self keeps evolving

Eivinas Butkus @eivinas.bsky.social · 23/07/2026

Thanks! I added it to our evidence pile.

010

Eivinas Butkus @eivinas.bsky.social · 23/07/2026

Thanks for sharing! This looks really interesting. I've put it down as a discussion point for the position paper.

010

Eivinas Butkus @eivinas.bsky.social · 23/07/2026

There's no strategic reason! Here are the other debates, which are a bit more international: 2026.ccneuro.org/gac-debates/

2026.ccneuro.org

Introducing the GAC Debates

010

Eivinas Butkus @eivinas.bsky.social · 23/07/2026

Thanks!! Definitely related to Platonic vs Umwelt representations.

000

Reposted by Eivinas Butkus

Dota Tianai Dong @dotadotadota.bsky.social · 21/07/2026

1/5 Over a decade of comparing deep neural networks to the human brain—but what have we actually learned? Our new @cp-trendscognsci.bsky.social Feature Review synthesizes a decade of brain–DNN comparisons, asking what they reveal about brain function across vision and language.

14421

Eivinas Butkus @eivinas.bsky.social · 21/07/2026

What do you think? Let us know in the replies! Kickoff workshop: Tuesday, August 4, 5:00 – 6:30 pm, Kimmel Center, Eisner & Lubin Auditorium, NYU #CCN2026

400

Eivinas Butkus @eivinas.bsky.social · 21/07/2026

The two camps: H1 (Proponents) Current evidence and methods support the claim that world models emerge in neural networks trained for prediction. H2 (Skeptics) Apparent world-model-like behavior is consistent with domain-specific heuristics and statistical shortcuts

130

Eivinas Butkus @eivinas.bsky.social · 21/07/2026

We'll explore whether standard self-supervised learning objectives of modern AI (such as next-token prediction) are sufficient to drive the emergence of world models. See full proposal here: sites.google.com/ccneuro.org/...

sites.google.com

GACs - 2026-2

Do World Models Emerge in Neural Networks Trained for Prediction? Eivinas Butkus, Columbia University Alexa Tartaglini, Stanford University Josh Ying, Columbia University Sonia Joseph, Meta Brian Che...

100

Eivinas Butkus @eivinas.bsky.social · 21/07/2026

Do world models emerge in neural networks trained for prediction? Generative Adversarial Collaboration (GAC) at #CCN2026

1275

Reposted by Eivinas Butkus

Paul Linton @lintonvision.bsky.social · 12/06/2026

@cogcompneuro.bsky.social Satellite Event: COMPUTATIONAL CONSCIOUSNESS SCIENCE Saturday, 1 August 2026 New York University Bringing computational modelling, neuroscience, and philosophy together to understand consciousness NYU Center for Mind, Brain and Consciousness #CCN2026

1197

Reposted by Eivinas Butkus

Lynn Sörensen @lynnkasorensen.bsky.social · 24/05/2026

There's one week left to apply to become this Fall's BCS Rising Star speaker 🌟 Please make sure to apply if you're a postdoc in the Brain & Cognitive Sciences🧠 Reposts are appreciated! 🪐

0118

Eivinas Butkus @eivinas.bsky.social · 18/05/2026

10/ Many thanks to Ilker Yildirim, Mario Belledonne, @peiyuchen.bsky.social, Jackie Gottlieb, Mike Woodford, @chrisbaldassano.bsky.social, @ruthrosenholtz.bsky.social, Xue-Xin Wei, Xaq Pitkow, and Alan Stocker for helpful discussions of earlier versions of this work.

000

Eivinas Butkus @eivinas.bsky.social · 18/05/2026

9/ We show how a cognitive function (attention), a neural mechanism (gain modulation), and a neurobiological constraint (metabolic cost) connect in a single mechanistic model to explain how selection and recurrence enable flexible, energy-efficient vision.

300

Eivinas Butkus @eivinas.bsky.social · 18/05/2026

8/ EAN also generalizes to classical attention tasks and captures canonical neural signatures of attention as emergent consequences of its top-down multiplicative gain modulation.

100

Eivinas Butkus @eivinas.bsky.social · 18/05/2026

7/ Beyond efficiency, EAN captures human error patterns in the visual-category-search task, and EAN's energy expenditure predicts participants’ subjective difficulty judgments beyond what stimulus parameters alone can explain.

100

Eivinas Butkus @eivinas.bsky.social · 18/05/2026

6/ The key result: in our “visual-category-search” task, attentional control costs around 4-5% but can save up to 50% of the total energy compared to a no-attention baseline.

100

Eivinas Butkus @eivinas.bsky.social · 18/05/2026

5/ We introduce EAN (Energy-efficient Attention Network), which implements attention through independent gain mechanisms over features, space, and time. EAN is optimized with a joint objective balancing task performance and energetic cost, including the cost of attention itself.

110

Eivinas Butkus @eivinas.bsky.social · 18/05/2026

4/ Attention requires its own machinery and energy to run. So does it actually bring *net* energetic benefits? To answer this, we account for all costs—action potentials, synaptic transmission, across all layers and time steps.

100

Eivinas Butkus @eivinas.bsky.social · 18/05/2026

3/ POSTER at VSS 2026: Tuesday, May 19, 8:30 am – 12:30 pm, Pavilion. Poster number 413. Come chat!

110

Eivinas Butkus @eivinas.bsky.social · 18/05/2026

2/ PREPRINT: with Josh Ying @zfjoshying.bsky.social and Niko Kriegeskorte “How attention saves energy in vision” biorxiv.org/content/10.6...

biorxiv.org

110

Eivinas Butkus @eivinas.bsky.social · 18/05/2026

1/ New preprint + VSS poster! Attention has long been thought to enable efficient vision. But does it? First demonstration that attention—consuming just 4-5% of the energy budget—can cut the energetic costs of vision in half. #VSS2026

1154

Eivinas Butkus @eivinas.bsky.social · 16/05/2026

9/ Beyond efficiency, EAN captures human error patterns in the visual-category-search task, and its energy expenditure predicts participants’ subjective difficulty judgments beyond what stimulus parameters alone can explain.

000

Reposted by Eivinas Butkus

Tobias Gerstenberg @tobigerstenberg.bsky.social · 03/04/2026

The Causality in Cognition Lab -- a supportive, bluesky-colored team -- is looking for a predoc to join us! Here are infos about the lab (cicl.stanford.edu) and the position (careersearch.stanford.edu/jobs/iriss-p...). The application deadline is May 1st. Please share, thank you 🙏

26536

Reposted by Eivinas Butkus

CogCompNeuro @cogcompneuro.bsky.social · 28/01/2026

🚨 #CCN2026 Proceedings submissions are open! CCN 2026 again features an 8-page Proceedings track (alongside extended abstracts). Accepted papers will appear in CCN-Proceedings (CCN‑P) with DOIs on OpenReview.

23321

Reposted by Eivinas Butkus

Erin Grant @eringrant.me · 06/12/2025

Thrilled to start 2026 as faculty in Psych & CS @ualberta.bsky.social + Amii.ca Fellow! 🥳 Recruiting students to develop theories of cognition in natural & artificial systems 🤖💭🧠. Find me at #NeurIPS2025 workshops (speaking coginterp.github.io/neurips2025 & organising @dataonbrainmind.bsky.social)

410928

Reposted by Eivinas Butkus

tyler bonnen @tylerbonnen.bsky.social · 25/11/2025

starting fall 2026 i'll be an assistant professor at @upenn.edu 🥳 my lab will develop scalable models/theories of human behavior, focused on memory and perception currently recruiting PhD students in psychology, neuroscience, & computer science! reach out if you're interested 😊

a red building on UPENN's campus photographed during the fall

the Philadelphia skyline, with clear skies and autumn trees

2522745

Eivinas Butkus @eivinas.bsky.social · 18/11/2025

21/ Poster #2510 (Fri 5 Dec, 4:30-7:30pm PT) at the main conference. Also at the Mechanistic Interpretability Workshop at NeurIPS (Sun 7 Dec). Come chat!

010

Eivinas Butkus @eivinas.bsky.social · 18/11/2025

20/ Thanks to the reviewers and the NeurIPS 2025 community. Thanks also to @zfjoshying, Yushu Pan, @eliasbareinboim (and the CausalAI Lab at Columbia University) for helpful feedback on this work.

110

Eivinas Butkus @eivinas.bsky.social · 18/11/2025

19/ Open questions: Do these findings extend to real-world LLMs, more complex causal structures? Could video models like Sora be learning physics simulators internally? We’re excited to explore this in future work.

110

Eivinas Butkus @eivinas.bsky.social · 18/11/2025

18/ Limitations: This is an existence proof in a constrained setting (linear Gaussian SCMs, artificial language). Let us know if you have ideas on how to test this in real LLMs. What may we be missing?

100

Eivinas Butkus @eivinas.bsky.social · 18/11/2025

17/ Broader implications: Even when trained using “statistical” prediction, neural networks can develop sophisticated internal machinery (compositional, symbolic structures) that support genuine causal models and reasoning. The “causal parrot” framing may be too limited.

100

Eivinas Butkus @eivinas.bsky.social · 18/11/2025

16/ Three lines of evidence that the model possesses genuine causal models: (1) it generalizes to novel structure-query combinations, (2) it learns decodable causal representations, and (3) representations can be causally manipulated with predictable effects.

100

Eivinas Butkus @eivinas.bsky.social · 18/11/2025

15/ Result 3 (continued): When we intervene on layer activations to change weight w_12 from 0 → 1, the model's predictions flip to match the modified causal structure. This suggests we’re decoding the actual underlying representation the model uses. (See paper for more quantitative results.)

100

Eivinas Butkus @eivinas.bsky.social · 18/11/2025

14/ Result 3: We can manipulate the model's internal causal representation mid-computation using gradient descent (following the technique from Li et al. 2023). Changing the SCM weights using the probe produces predictable changes in the network’s outputs.

100

Eivinas Butkus @eivinas.bsky.social · 18/11/2025

13/ Result 2: We can decode the SCM weights directly from the transformer's residual stream activations using linear and MLP probes. The model builds interpretable internal representations of causal structure.

100

Eivinas Butkus @eivinas.bsky.social · 18/11/2025

12/ But does the model really have internal causal models, or is this just lucky generalization? We probe inside to find out...

100

Eivinas Butkus @eivinas.bsky.social · 18/11/2025

11/ Result 1: Yes! The model generalizes to counterfactual queries about D_test SCMs, reaching near-optimal performance. It must have: (1) learned a counterfactual inference engine, (2) discovered D_test structures from DATA strings, (3) composed them together.

100

Eivinas Butkus @eivinas.bsky.social · 18/11/2025

10/ The generalization challenge: We hold out 1,000 SCMs (D_test) where the model sees *only* interventional DATA strings during training and zero INFERENCE examples. Can it still answer counterfactual queries about these SCMs?

100

Eivinas Butkus @eivinas.bsky.social · 18/11/2025

9/ We train a GPT-style transformer to predict the next token in this text. The key question: does it simply memorize the training data, or does it discover causal structure and perform inference?

110

Eivinas Butkus @eivinas.bsky.social · 18/11/2025

8/ Our setup: ~59k SCMs, each defined by 10 ternary weights. We generate training data using the SCMs in an artificial language with two string types: (1) DATA provide noisy interventional samples and (2) INFERENCE — counterfactual means/stds.

100

Eivinas Butkus @eivinas.bsky.social · 18/11/2025

7/ Our hypothesis: next-token prediction can drive the emergence of genuine causal models and inference capabilities. We tested this in a controlled setting with linear Gaussian structural causal models (SCMs).

100

Eivinas Butkus @eivinas.bsky.social · 18/11/2025

6/ Pearl (Amstat News interview, 2023) and Zečević et al. (2023) acknowledge causal info exists in text, but argue LLMs are merely "causal parrots"—they memorize and recite but do not possess actual causal models.

100

Eivinas Butkus @eivinas.bsky.social · 18/11/2025

5/ But natural language used to train LLMs contains rich descriptions of interventions and causal inferences. Passive data != observational data. Text has L2/L3 information!

100

Eivinas Butkus @eivinas.bsky.social · 18/11/2025

4/ Pearl’s vivid metaphor from “The Book of Why” (p. 362): “Like prisoners in Plato's cave, deep-learning systems explore shadows on the wall and learn to predict their movements. They lack understanding that shadows are mere projections of 3D objects in 3D space.”

110

Eivinas Butkus @eivinas.bsky.social · 18/11/2025

3/ Pearl's argument: DNNs trained on “passive” observations using statistical prediction objectives are fundamentally limited to associations (L1) and cannot reason about interventions (L2) or counterfactuals (L3).

100

Eivinas Butkus @eivinas.bsky.social · 18/11/2025

2/ Paper: “Causal Discovery and Inference through Next-Token Prediction” (Butkus & Kriegeskorte) OpenReview: openreview.net/pdf?id=MMYTA..., NeurIPS: neurips.cc/virtual/2025...

110

Eivinas Butkus @eivinas.bsky.social · 18/11/2025

1/ Can causal models and causal inference engines emerge through next-token prediction? Judea Pearl and others (Zečević et al. 2023) have argued no. We present behavioral and mechanistic evidence that this is possible. #neurips2025 #NeurIPS

181