Sign in

Andrew Saxe

@saxelab.bsky.social
4.4K followers 483 following 47 posts

Professor at the Gatsby Unit and Sainsbury Wellcome Centre, UCL, trying to figure out how we learn

PostsRepliesMedia
Reposted by Andrew Saxe
Kanishka Misra @kanishka.bsky.social · 09/09/2026
Understanding how learners conclude “X laughed Y” is incorrect is an age-old question, with several hypotheses, some of which have been ~impossible to disentangle! @tomyxw.bsky.social, @fredashi.bsky.social, and I use controlled rearing to shed light on this in our new EMNLP paper: 1/n
Title slide for “Disentangling Statistical Preemption from Entrenchment in Language Models’ Avoidance of Overgeneralization,” by Yixuan Wang, Freda Shi, and Kanishka Misra. Includes the main results plot and experimental design.
2179
Reposted by Andrew Saxe
Laura Driscoll @lndriscoll.bsky.social · 08/08/2026
Confused about all this talk of compositionality in neuroscience? Read our new perspective www.nature.com/articles/s41... with authors Reidar Riveland and Alex Pouget.
nature.com
The compositionality continuum as a principle for studying the neural basis of intelligence - Nature Neuroscience
Compositionality exists on a continuum of increasing complexity rather than as a binary trait. Studying its implementation in simpler biological and artificial systems is the most tractable path towar...
212043
Reposted by Andrew Saxe
Reza Shadmehr @rezashadmehr.bsky.social · 28/07/2026
If an action results in error, each neuron requires an individualized teaching signal that guides change in its output. This is the credit assignment problem of learning. Are there neurons in the brain that can compute such a sophisticated teaching signal? Yes. www.biorxiv.org/content/10.6...
biorxiv.org
Climbing fibers encode the gradient of a loss function for the cerebellum
Neurons in the brain are often many synapses away from motoneurons, yet if a movement results in error, each distant neuron needs a teacher that considers its specific contribution to production of th...
218170
Reposted by Andrew Saxe
Samuel Lippl @sflippl.bsky.social · 08/07/2026
Pretraining + fine-tuning powers modern ML, but we lack a theoretical understanding of how pretraining actually shapes downstream learning. In our new @icmlconf.bsky.social paper, we address this gap! 📅 July 9th, Poster #4502 Session 8! 🧵 arxiv.org/pdf/2602.20062
Diagram with "l-order" on the x-axis and "Pretraining dependence" on the y axis. The diagram highlights that only the upper right triangle is a possible region and highlights four distinct regimes: (I) a rich, pretraining independent regime, (II) a lazy, pretraining-dependent regime, (III) lazy, pretraining-independent regime, and (IV) a rich, pretraining dependent regime. Different initialization parameters move us between these regimes.
1193
Reposted by Andrew Saxe
Athena Akrami @athenaakrami.bsky.social · 06/07/2026
The lab is at @fens.org, with 5 posters — spatial & auditory working memory, prefrontal decision-making, dopamine in statistical learning, and a 2-photon rat-brain atlas for BrainGlobe. Unfortunately, I couldn't make it this year, but come find the rest of the lab on Tue, Thurs, & Fri! #FENS2026
Flyer — LIM Lab (Akrami Lab, Sainsbury Wellcome Centre / UCL) posters at FENS Forum 2026, Barcelona, 6–10 July. Tue 7 July board 129 (Arpit Agarwal); Thu 9 July boards 639 (Audra Rybak), 641 (Kyunghye Lee), 022 (Viktor Plattner); Fri 10 July board 353 (Lida Pentousi).
2347
Reposted by Andrew Saxe
Yedi Zhang @yedizhang.bsky.social · 02/07/2026
Is Muon as good as they say? We looked beyond training speed and found a hidden cost: Muon loses the simplicity bias of older optimizers like gradient descent — and this matters for generalization. Led by Sara Dragutinović and advised by Rajesh Ranganath arxiv.org/abs/2603.00742
1102
Reposted by Andrew Saxe
Neil Burgess @neilburgess10.bsky.social · 01/07/2026
Ever wondered how the hippocampal cognitive map is read-out? @changmin-yu.bsky.social @zilong-ji.bsky.social with Jake & John, show that, during navigation, theta sweeps indicate remembered goal-directions (cf. current/next movements or perceptual targets) 1/2 www.nature.com/articles/s41...
Decoded locations from place cell firing sweep towards a remembered goal location during each theta cycle irrespective of the current movement or head direction
710047
Reposted by Andrew Saxe
Ezekiel Williams @ezekielwilliams.bsky.social · 10/06/2026
1/7 Excited to share my last PhD article, just accepted to ICML 2026! In it, we (me, Alexandre Payeur, Guillaume Lajoie) used dynamical systems theory to study "local" learning in linear recurrent neural networks. See link for the paper, and thread for a brief summary. arxiv.org/abs/2606.00243
arxiv.org
Dynamics and Representation Structure of Local Approximations to Gradient-Based Learning in Linear Recurrent Neural Networks
Biological and neuromorphic recurrent neural networks (RNNs) are subject to spatial and temporal locality constraints on the information that can plausibly be used during learning. A common strategy t...
34916
Reposted by Andrew Saxe
Yedi Zhang @yedizhang.bsky.social · 23/04/2026
Come chat about this @iclr-conf.bsky.social! Friday 3:15 PM, Pavilion 4, Poster #4216
062
Reposted by Andrew Saxe
Anna Schapiro @annaschapiro.bsky.social · 13/04/2026
We’ve got an exciting new thing to share! We have causal evidence (using TMR) that memory reactivation during sleep promotes abstract understanding of underlying structure, allowing transfer learning in a new domain with zero superficial feature overlap with the learned one.
112135
Reposted by Andrew Saxe
Athena Akrami @athenaakrami.bsky.social · 13/04/2026
New preprint! 🧠 How do RNNs learn abstract rules from sequences, independent of specific stimuli? By Vezha Boboeva, with Alberto Pezzotta & George Dimitriadis "From sequences to schemas: low-rank recurrent dynamics underlie abstract relational representations" www.biorxiv.org/content/10.6...
111137
Reposted by Andrew Saxe
Stefano Sarao Mannelli @stefsm.bsky.social · 10/04/2026
Two Analytical Connectionism-related updates: 1. ⏰ 1 week left to apply! Interested in language + AI & cognition? Don’t miss it: www.analytical-connectionism.net/school/2026/ 2. 📜 Lecture notes from the first two editions are finally out: proceedings.mlr.press/v320/
analytical-connectionism.net
2026 School on Analytical Connectionism
A 2-week summer course hosted at Chalmers University of Technology on analytical approaches to language acquisition and higher-level cognition.
082
Reposted by Andrew Saxe
Sainsbury Wellcome Centre @sainsburywellcome.bsky.social · 13/02/2026
We’re hiring a Group Leader! Join us to lead a transformative initiative in human systems neuroscience. Find out more and apply ⤵️ www.sainsburywellcome.org/content/curr...
We're hiring! This is a unique opportunity to translate our understanding of neural computation - from circuit-level mechanisms to computational principles -  into the human brain, through the establishment of cutting-edge human neural recording capabilities with collaborators in London and abroad.
13526
Andrew Saxe @saxelab.bsky.social · 02/04/2026
Postdoc opening! Come work with us on deep learning theory relevant to AI safety Deadline: 7 Apr 2026 Details and application: www.ucl.ac.uk/work-at-ucl/...
ucl.ac.uk
UCL – University College London
UCL is consistently ranked as one of the top ten universities in the world (QS World University Rankings 2010-2022) and is No.2 in the UK for research power (Research Excellence Framework 2021).
2186
Andrew Saxe @saxelab.bsky.social · 02/04/2026
Very excited by this year's Analytical Connectionism Summer School! A dream lineup of speakers on the topic of language acquisition in minds and machines Bursaries available to cover costs Aug 17 – Aug 28, 2026 Gothenburg Details: www.analytical-connectionism.net//school/2026/
0172
Reposted by Andrew Saxe
Blake Richards @tyrellturing.bsky.social · 27/03/2026
A great entry into the proposals available for physiologically plausible gradient descent! I think the way they use dendrite targeting inhibition in this model is particularly elegant. Time to start testing these ideas folks!!! #neuroscience 🧪 #NeuroAI
0339
Reposted by Andrew Saxe
bioRxiv Neuroscience @biorxiv-neursci.bsky.social · 23/03/2026
The First 1,000 Days (1kD) Project - Collecting and Analyzing an Ultra-Dense Naturalistic Dataset of Human Baby Development www.biorxiv.org/content/10.64898/20…
052
Reposted by Andrew Saxe
Francis Bach @bachfrancis.bsky.social · 05/03/2026
Looking for alternatives to quadratic functions for closed-form analysis in optimization? This post explores matrix Riccati dynamics and their applications to neural networks. francisbach.com/closed-form-...
0192
Reposted by Andrew Saxe
Blake Richards @tyrellturing.bsky.social · 19/03/2026
Here's a lovely #blueprint on a new study from our lab led by @royeyono.bsky.social. tl;dr: it implies that there may be interneurons whose role is to normalize credit assignment signals during learning. #neuroscience 🧪
25012
Reposted by Andrew Saxe
Pascal Mamassian @mamassian.bsky.social · 12/03/2026
A new Department of Cognitive Science is being created at Bocconi University in Milan, Italy. Here is the call for a cluster hire (for around 10 faculty) in all areas of cognitive science, at both junior and senior levels: www.unibocconi.it/en/faculty-a... Deadline: May 4th, 2026
unibocconi.it
Open Rank Faculty Cluster Hire Search for the New Department of Cognitive Science at Bocconi - Bocconi University
3150119
Reposted by Andrew Saxe
Andrew MacAskill @macaskillaf.bsky.social · 12/03/2026
Poster tonight at #cosyne26 (1-079)! @wanqingjiang.bsky.social & @noehamou.bsky.social show that mice learn hidden community structure in a 15-odour graph even when transition statistics are flat. Fun collaboration with @saxelab.bsky.social that started with East London coffees ☕!
1345
Reposted by Andrew Saxe
Sam Gershman @gershbrain.bsky.social · 11/03/2026
Really neat work by Fountas and colleagues at UCL: arxiv.org/abs/2603.04688 They propose that consolidation reflects a form of "predictive forgetting" that aids generalization.
arxiv.org
Why the Brain Consolidates: Predictive Forgetting for Optimal Generalisation
Standard accounts of memory consolidation emphasise the stabilisation of stored representations, but struggle to explain representational drift, semanticisation, or the necessity of offline replay. He...
39332
Reposted by Andrew Saxe
Athena Akrami @athenaakrami.bsky.social · 10/03/2026
Thanks @natmesanash.bsky.social for covering our new work, in @thetransmitter.bsky.social!
14011
Reposted by Andrew Saxe
Ruairidh McLennan Battleday @battleday.bsky.social · 05/03/2026
📢📢 Announcing this year's conference on the Mathematics of Neuroscience & AI (Rome, 9-12th June). We’ve got a stellar line-up and venue, and invite everyone to join: www.neuromonster.org
23017
Reposted by Andrew Saxe
Gatsby Computational Neuroscience Unit @gatsbyucl.bsky.social · 25/02/2026
📢 Job alert - Deep Learning Theory & AI Safety Applications open for a postdoc fellow (@saxelab.bsky.social lab) to study artificial deep networks using techniques from applied maths & stat physics. ⏰ Deadline: 26 Mar 2026 🤝 In collaboration with @stefsm.bsky.social ℹ️ www.ucl.ac.uk/life-science...
0104
Reposted by Andrew Saxe
Laura Grima @lauragrima.bsky.social · 24/02/2026
Excited to be co-organising a #cosyne2026 workshop with Alison Comrie on 'algorithms for learning from scratch'! With a great line-up of speakers, we'll be tackling the question of what processes enable naive biological & artificial agents to adapt to new situations. Info here: tinyurl.com/4u8enf7k
sites.google.com
learningfromscratch
march 16th, workshop day 1 @ cosyne 2026
15017
Reposted by Andrew Saxe
Stefano Sarao Mannelli @stefsm.bsky.social · 18/02/2026
📢 We’re now accepting applications for the 2026 School on Analytical Connectionism dedicated this year to Language Acquisition. 📍 Gothenburg, Sweden 🗓️ August 17–28, 2026 ☠️ Apply by April 17! 🔗 analytical-connectionism.net/school/2026/ 👇 Meet the experts joining us this summer!
1208
Reposted by Andrew Saxe
Athena Akrami @athenaakrami.bsky.social · 16/02/2026
Thrilled to finally share this work! 🧠🔊 Using a new reinforcement-free task we show mice (like humans) extract abstract structure from sound (unsupervised) & dCA1 is causally required by building factorised, orthogonal subspaces of abstract rules. Led by Dammy Onih! www.biorxiv.org/content/10.6...
biorxiv.org
315752
Andrew Saxe @saxelab.bsky.social · 16/02/2026
How to apply: Salary: USD 80,000–100,000 (50-74k GBP) annualised Initial contract: 6 months, w/ extension based on funding Details: docs.google.com/document/d/1... Application: forms.gle/xKukH74iX16p... 4
docs.google.com
[Hiring] Principia Research Fellows
Principia Research Fellows: Theoretical Model Organisms for AI Safety Principia · London · Fixed-term (6 months) with potential extension · Starting ASAP We are launching Principia, a new technical re...
030
Andrew Saxe @saxelab.bsky.social · 16/02/2026
We’re hiring postdocs/research scientists! Your interests can be anywhere on the spectrum from pure theory to empirically testing predictions relevant to AI safety. Our theoretical work relies on dynamical systems and tools from statistical physics. 3
151
Andrew Saxe @saxelab.bsky.social · 16/02/2026
We avoid many unwanted outcomes in the physical world using our knowledge of physics, and basic deep learning theory should eventually enable the same for AI. We focus on simple, analytically tractable “model organisms” that capture essential learning dynamics and behaviours. 2
140
Andrew Saxe @saxelab.bsky.social · 16/02/2026
Excited to launch Principia, a nonprofit research organisation at the intersection of deep learning theory and AI safety. Our goal is to develop theory for modern machine learning systems that can help us understand complex network behaviors, including those critical for AI safety and alignment. 1
19428
Reposted by Andrew Saxe
SueYeon Chung @sueyeonchung.bsky.social · 10/02/2026
Our paper is out in @natneuro.nature.com! www.nature.com/articles/s41... We develop a geometric theory of how neural populations support generalization across many tasks. @zuckermanbrain.bsky.social @flatironinstitute.org @kempnerinstitute.bsky.social 1/14
7278101
Andrew Saxe @saxelab.bsky.social · 05/02/2026
A great question, I'm not sure. It's important to understand if muon shares similar inductive biases.
010
Andrew Saxe @saxelab.bsky.social · 05/02/2026
I agree, there seem to be connections but it's not fully clear to me why. SLT is a static theory, and yet Daniel Murfet and others have shown that the stages we see also correspond to SLT posteriors of increasing complexity.
110
Andrew Saxe @saxelab.bsky.social · 03/02/2026
Upcoming online talk next Monday 9th February, at the ELLIS Reading Group on Mathematics & Efficiency of Deep Learning! Open to all. Info at sites.google.com/view/efficie...
sites.google.com
DLMath&Efficiency
This reading group examines the interplay between the theoretical foundations of deep learning and the practical challenge of making machine learning efficient. On the theory side, we study mathematic...
040
Andrew Saxe @saxelab.bsky.social · 03/02/2026
Equipped with this theory, we make new predictions about how network width, data distribution, and initialization affect learning dynamics. For example, increasing the number of attention heads in linear attention shortens the plateaus in learning.
160
Andrew Saxe @saxelab.bsky.social · 03/02/2026
So when progressing simple -> complex, linear networks learn solutions of increasing rank, ReLU networks learn solutions with increasing kinks, convolutional networks learn solutions with increasing convolutional kernels, and attention models learn solutions with increasing heads.
160
Andrew Saxe @saxelab.bsky.social · 03/02/2026
Here the notion of simplicity is the number of effective units in the architecture: hidden neurons, convolutional kernels, or attention heads.
150
Andrew Saxe @saxelab.bsky.social · 03/02/2026
Finally, we demonstrate gradient descent sometimes naturally evolves along the connecting paths between saddles iteratively, yielding saddle-to-saddle dynamics. We identify two distinct mechanisms: timescale separation between directions or units, depending on the architecture.
180
Andrew Saxe @saxelab.bsky.social · 03/02/2026
We then show that saddles are connected by gradient descent paths (invariant manifolds). Along these paths, a larger network behaves like a smaller one, retaining the same simplicity during a saddle-to-saddle transition.
182
Andrew Saxe @saxelab.bsky.social · 03/02/2026
We first show that saddle points are ubiquitous in the loss landscape: fixed points of smaller networks can be embedded as saddle points of larger networks, yielding a nested hierarchy of saddles. These saddles exist in any network that contains a sum of repeated units.
1100
Andrew Saxe @saxelab.bsky.social · 03/02/2026
We present a theoretical framework that explains dynamical simplicity bias arising from saddle-to-saddle learning dynamics across neural network architectures: Fully-connected, convolutional, attention-based, and more. yedizhang.github.io/simplicity
yedizhang.github.io
Saddle-to-Saddle Dynamics Explains A Simplicity Bias Across Neural Network Architectures
We define a notion of simplicity that applies to a broad class of architectures, and show both analytically and with explicit examples that during learning simplicity decreases in a step-wise manner.
182
Andrew Saxe @saxelab.bsky.social · 03/02/2026
Why don’t neural networks learn all at once, but instead progress from simple to complex solutions? And what does “simple” even mean across different neural network architectures? Sharing our new paper @iclr_conf led by Yedi Zhang with Peter Latham arxiv.org/abs/2512.20607
716042
Reposted by Andrew Saxe
Gatsby Computational Neuroscience Unit @gatsbyucl.bsky.social · 15/01/2026
📢 Applications open on 19 Jan for the 7-week #Mathematics #SummerSchool in London. You will develop the maths skills and intuition necessary to enter the #TheoreticalNeuroscience / #MachineLearning field. Find out more & register for the information webinar 👉 www.ucl.ac.uk/life-science...
Applications for 2026 entry to the Gatsby Bridging Programme (7-week maths summer school) will open on 19 Jan and close on 16 Feb. Designed for students who wish to pursue a postgrad research degree in theoretical neuroscience or foundational machine learning but whose degree programme lacks a strong maths focus. Applications from students in underrepresented groups in STEM strongly encouraged. A small number of bursaries available.
Register for the information webinar on 23 Jan.
12427
Reposted by Andrew Saxe
Anna Schapiro @annaschapiro.bsky.social · 15/01/2026
Really thrilled that this paper led by @neurozz.bsky.social is now published in its final version in @elife.bsky.social!! This is a memory-focused (as opposed to RL-focused) account of the detailed characteristics of forward and backward awake and sleep replay! elifesciences.org/articles/99931
elifesciences.org
A unifying account of replay as context-driven memory reactivation
A context-driven memory model simulates a wide range of characteristics of waking and sleeping hippocampal replay, providing a new account of how and why replay occurs.
314153
Reposted by Andrew Saxe
Blake Richards @tyrellturing.bsky.social · 14/01/2026
Our paper on the "Oneirogen hypothesis" is now up in its revised form on eLife! This is the hypothesis that psychedelics induce a dream-like state, which we show via modelling could explain a variety of perceptual and learning effects from such drugs. elifesciences.org/reviewed-pre... 🧠📈 🧪
elifesciences.org
The oneirogen hypothesis: modeling the hallucinatory effects of classical psychedelics in terms of replay-dependent plasticity mechanisms
26014
Reposted by Andrew Saxe
Jeff Johnston @wjj.bsky.social · 09/01/2026
By the way, if you’re interested in working together on problems like this, I’m starting my lab at UCSF this summer. Get in touch if you’re interested in doing a postdoc! More info here: wj2.github.io/postdoc_ad (7/7)
wj2.github.io
W. Jeffrey Johnston - Postdoctoral position ad
12914
Reposted by Andrew Saxe
Armin Lak @laklab.bsky.social · 05/01/2026
New preprint. We show that in addition to reward prediction errors (RPEs), dorsal striatal dopamine signals encode sensory prediction errors (SPEs), the difference between sensory prior & observed stimulus. www.biorxiv.org/content/10.6...
biorxiv.org
Dorsal striatal dopamine integrates sensory and reward prediction errors to guide perceptual decisions
Perceptual decisions are shaped by expectations about sensory stimuli and rewards, learned through sensory and reward prediction errors. Dopamine is known to convey reward prediction errors that shape...
38626
Reposted by Andrew Saxe
Tim Verstynen @tdverstynen.bsky.social · 22/12/2025
Sleep dependent consolidation and replay that doesn’t require the hippocampus? Very beautiful work by Marcus Stephenson-Jones’ lab on sleep driven sequential skill consolidation in the striatum. www.biorxiv.org/content/10.1...
biorxiv.org
03010