iiinova — where your digital self keeps evolving

Jascha Achterberg @achterbrain.bsky.social · 09/03/2026

Going to Cosyne? There will be a "Biologically-inspired Artificial Intelligence: Challenges and Opportunities" workshop on Monday 16th! 🧠 Exciting lineup of speakers alongside the opportunity to present your #NeuroAI poster, more info on program & poster signup below! #compneuro #neuroscience

2478 0

Jascha Achterberg @achterbrain.bsky.social · 21/11/2025

We also find that while complex regions are needed to learn complex tasks, these tasks are eventually moved toward simpler regions, similar to how you may struggle the first time when learning a new skill, but slowly get better with practice.

110 0

Jascha Achterberg @achterbrain.bsky.social · 21/11/2025

Furthermore, we find that these pathways mirror our expected behavior of pathways in the brain! We find that difficult tasks need to be learned in more complex regions, similar to how you need to think “harder” when learning how to solve a difficult math problem.

110 0

Jascha Achterberg @achterbrain.bsky.social · 21/11/2025

With these three features in place, we find that our third criterion of distinct pathways is also met. While baseline models exhibit largely random expert usage patterns, our models exhibit highly structured pathways between regions that reliably emerge during learning.

110 0

Jascha Achterberg @achterbrain.bsky.social · 21/11/2025

Our third contribution is expert dropout. Without this feature, we find models suffer large performance deficits when experts outside of the active pathway are disabled. However, we would want models to be primarily dependent on the experts that are most being used.

110 0

Jascha Achterberg @achterbrain.bsky.social · 21/11/2025

When put together, these two contributions resulted in remarkable pathway consistency in our model, which we measured by correlating the routing patterns across 10 different models trained on the same tasks.

110 0

Jascha Achterberg @achterbrain.bsky.social · 21/11/2025

We then identify three inductive biases that yield pathways that meet each of these criteria. The first of these is a routing loss that penalizes the use of more complex experts during training, and the second scales this loss by the model’s performance on the task being solved.

110 0

Jascha Achterberg @achterbrain.bsky.social · 21/11/2025

We first needed to create a model in which we could study pathway formation. We chose a Heterogeneous Mixture-of-Experts architecture, in which information can be dynamically routed to computational experts, or regions, of varying sizes. We train model on 82 tasks of varying complexity (ModCog)!

110 0

Jascha Achterberg @achterbrain.bsky.social · 21/11/2025

Brains have many pathways / subnetworks but which principles underlie their formation? In our #NeurIPS paper lead by Jack Cook we identify biologically relevant inductive biases that create pathways in brain-like Mixture-of-Experts models🧵 #neuroskyence #compneuro #neuroAI arxiv.org/abs/2506.02813

13712 0

Jascha Achterberg @achterbrain.bsky.social · 29/05/2025

*Hierarchical choice code* Temporal cross-correlation analysis revealed hierarchical coding of problem structure across all regions, with most regions being driven by temporal similarity of time windows across choices, with vlPFC also responding for the repeated order of operations across choices.

100 0

Jascha Achterberg @achterbrain.bsky.social · 29/05/2025

*Move codes* Next, analysing dynamics in the "Move space", we found: Move coding develops first in vlPFC before reaching dPM; other regions showed weaker move coding. "Move space" generally orthogonal to "Goal space" (except in dmPFC).

100 0

Jascha Achterberg @achterbrain.bsky.social · 29/05/2025

*Goal and location codes* We projected neural activity into the "Goal space" & measured distances between projections grouped by current position vs. goal. We saw regional specialization: vlPFC driven by location; dmPFC more driven by goal (maintained throughout trial); dPM & I/O with mixed code.

100 0

Jascha Achterberg @achterbrain.bsky.social · 29/05/2025

*Analysing neural subspaces* To understand frontal cortex computations, we identified neural subspaces in relation to key variables and studied the population dynamics over the duration of the trial. For each region, we asked: Which variables drive the shape & dynamics of projections?

100 0

Jascha Achterberg @achterbrain.bsky.social · 29/05/2025

*Recordings* We recorded 1374 neurons from four frontal regions with semi-chronic microelectrode arrays: ventrolateral, dorsal premotor, dorsomedial, insula/orbitofrontal. This wide coverage across frontal cortex in the same complex task makes this a unique dataset in monkey electrophysiology.

110 0

Jascha Achterberg @achterbrain.bsky.social · 29/05/2025

*Routes through the maze* Depending on available choice options, goal locations can be reached via 2-step or 4-step routes.

100 0

Jascha Achterberg @achterbrain.bsky.social · 29/05/2025

*Task* Monkeys solve a multi-step maze task which requires them to navigate a 2D grid from the start location to 1 of 4 goal locations, using saccades. Monkeys start at the center, need to remember the goal location (presented at the start of the trial), and navigate using presented choice options.

100 0

Jascha Achterberg @achterbrain.bsky.social · 08/11/2024

Plenty of training programs now put a focus on Open Science. We find that our respondents already show strong experience with open source and open sharing of science!

100 0

Jascha Achterberg @achterbrain.bsky.social · 08/11/2024

We do find that the training needs of respondents are extremely heterogenous, with both methods and theory across both neuro and AI being the gap that some group of respondents wanted to close in their own education.

110 0

Jascha Achterberg @achterbrain.bsky.social · 08/11/2024

What surprised us was that most respondents hoped to find a position that allows to combine academia with industry but only few thought they were likely to achieve their goal. This might inform both how we plan courses but also how we think about hiring in the future!

100 0

Jascha Achterberg @achterbrain.bsky.social · 08/11/2024

There’s been some discussion in the community whether NeuroAI should be about “Neuro to AI”-transfer or vice versa. We find that both are topics of great interest!

241 0

Jascha Achterberg @achterbrain.bsky.social · 08/11/2024

First of all, why are students motivated to work on the intersection of neuroscience and AI in the first place? Is it just a career move, based on AI being a hot topic? No! We find curiosity is much more important than career-focused drivers.

100 0

Jascha Achterberg @achterbrain.bsky.social · 08/11/2024

Here Andrea Luppi and I + lots of collaborators (see below) survey current trainees on the intersection of AI / neuro to learn how trainees think they could be best supported! Some highlighted findings in thread, with lots more in the paper:

100 0

Jascha Achterberg @achterbrain.bsky.social · 08/11/2024

Last year Zador et al called to train a new generation of researchers, equally at home in CompSci & #neuroscience to accelerate our understanding of the nature of intelligence, but how can we get this interdisciplinary training right? 🧵 #MLsky #NeuroAI #CompNeuro www.nature.com/articles/s41...

1377 0

Jascha Achterberg @achterbrain.bsky.social · 09/09/2024

Join our ARIA-funded project as a postdoc on brain-inspired computing 🤖🧠, at Imperial College London! Super exciting opportunity connecting both fundamental research and the creation of cutting-edge technologies! #neuroscience #MLsky #NeuroAI #CompNeuro www.imperial.ac.uk/jobs/search-...

030 0

Jascha Achterberg @achterbrain.bsky.social · 03/06/2024

Starting next week I will be in the US 🇺🇸, until mid AUG. First in New York, later SF & Boston! If you want to meet to discuss (or know of a meet-up on) efficient AI, dynamic representations, NeuroAI, hardware accelerators, & neuromorphic, please reach out! #neuroscience #MLsky #NeuroAI #CompNeuro

030 0

Jascha Achterberg @achterbrain.bsky.social · 02/05/2024

PyTorch user guides offering advice for some very specific life situations -- nothing like a little hidden side comment to remind us all that these documentations and guides are all written by the community! via pytorch.org/tutorials/ad... #MLSky #compneuro #neuroAI #deeplearning

121 0

Jascha Achterberg @achterbrain.bsky.social · 01/02/2024

Amazing looking preprint from Tim Buschman's group: 'Building compositional tasks with shared neural subspaces' www.biorxiv.org/content/10.1... #neuroscience #compneuro

0135 0

Jascha Achterberg @achterbrain.bsky.social · 20/11/2023

🪢 Lastly, we find that all these findings arise in unison in seRNNs, highlighting that this diverse set of seemingly unrelated brain features can result from a shared underlying optimisation process!

100 0

Jascha Achterberg @achterbrain.bsky.social · 20/11/2023

⚡️ While the connectome of seRNNs is modular & the neurons are spatially organised by their code, we find that seRNNs still show an information-rich mixed selective code, that also is commonly found in frontal cortex. seRNNs achieve this using very efficient neuronal activations!

100 0

Jascha Achterberg @achterbrain.bsky.social · 20/11/2023

📡 To use our model system to jointly study structural and functional phenomena, we also analyse the code used by neurons in the network. Similar to brains, we find that information communication & processing in seRNNs is spatially structured!

110 0

Jascha Achterberg @achterbrain.bsky.social · 20/11/2023

🕸️ Looking at the network structure more specifically, we see that seRNNs develop a highly modular connectome with strong small-world characteristics. These features are commonly found in biology and thought to guide efficient information processing!

110 0

Jascha Achterberg @achterbrain.bsky.social · 20/11/2023

🧠 Even in the simplest observation we already see that an isolated seRNN develops connection patterns commonly observed in brains – here we see a relationship between connection strength and spatial distance and clustering of connections in the connectome!

110 0

Jascha Achterberg @achterbrain.bsky.social · 20/11/2023

🔧 All our tests of spatially embedded RNNs (we call them seRNNs) are made in comparison with standard L1 regularised models, matched for sparsity. We train a large batch of networks with varying regularisation strengths.

110 0

Jascha Achterberg @achterbrain.bsky.social · 20/11/2023

🌐 Starting from standard RNNs, optimised for a working memory task, we create a spatial embedding by regularising them by distance in 3D Euclidean space & nudge them to prioritise highly communicative connections (Topological, lots of theory on this in paper!)

110 0

Jascha Achterberg @achterbrain.bsky.social · 03/11/2023

Super excited to have contributed to this massive effort in reviewing the literature and current trends in representational alignment of biological and artificial systems! Amazing team, expertly lead by Lukas and Ilia! arxiv.org/pdf/2310.130... #neuroscience #compneuro #mlsky #neuroai

Screenshot of first page of https://arxiv.org/pdf/2310.13018.pdf

0165 0

Jascha Achterberg @achterbrain.bsky.social · 19/10/2023

If you are interested in neuroscience, AI or hardware optimisation, you will want to have a look at this paper published today, on a new brain-inspired AI accelerator design: Neural inference at the frontier of energy, space, and time www.science.org/doi/10.1126/... #neuroscience #compneuro #mlsky

Axiomatic design

NorthPole, an architecture and a programming model for neural inference, reimagines (Fig. 1) the interaction between compute and memory by embodying 10 interrelated, synergistic axioms that build on brain-inspired computing (14, 15). Within the broad domain of neural inference, NorthPole co-optimizes architecture, algorithms, and software and enables a simple input-output model (16).

051 0

Jascha Achterberg @achterbrain.bsky.social · 17/10/2023

If you haven't come across it, "Two views on the cognitive brain", discussing Sharringtonian vs Hopfieldian views on brain computations, is very much worth reading! Really enjoyed @dlbarack.bsky.social & John's presentations at CCNeuro! www.nature.com/articles/s41... #neuroscience #compneuro

12913 0

Jascha Achterberg @achterbrain.bsky.social · 16/10/2023

Useful research paper for a Monday morning: "Systematically Improving Espresso: Insights from Mathematical Modeling and Experiment" ☕️🌞 www.sciencedirect.com/science/arti... #AcademicSky

1132 0

Jascha Achterberg @achterbrain.bsky.social · 09/10/2023

Super cool new open online course on neuroscience for people with a machine learning background, developed by Dan Goodman and Marcus Gosh! neuro4ml.github.io #neuroscience #compneuro #MLsky

12115 0

Jascha Achterberg @achterbrain.bsky.social · 28/09/2023

One of my favourites is in link.springer.com/article/10.1... The authors thank their neighbour Gary for chasing down a burglar who stole the computer with the unsubmitted manuscript! #academicsky

Text on image: The majority of the work presented here was undertaken as part of C. Sharpe’s honours year at the University of Sydney, but was originally conceived by G. F. Lewis. We would like to thank A. Arbey et al. for making their AlterBBN code publicly available as this paper would not have been possible without it. G. F. Lewis received no funding to support this research. C. Sharpe would also like to thank NSW police for ensuring a swift return of his belongings, including his laptop, after having them burgled from his house a few weeks before this paper’s submission. Additionally, he would like to thank his neighbour, Gary, who spotted the burglar and, rather than simply phoning the police and staying put, decided to yell ‘you better run fast mate’ before chasing the man down the street, tackling him, pinning him to the ground, and then calling the police, all while still in his pyjamas and a sleepy daze.

1112 0

Jascha Achterberg @achterbrain.bsky.social · 24/09/2023

Another fun overview of higher cognition in non-primate species: "Neuroscience of cognitive control in crows" www.cell.com/trends/neuro... #neuroscience

screenshot of graphic and abstract of linked paper

150 0

Jascha Achterberg @achterbrain.bsky.social · 23/09/2023

Thomas Akam is starting his group at Oxford & gave an overview of his work last week. So many cool past & future projects, one that I particularly liked, on complementary task representations in PFC and hippocampus: www.nature.com/articles/s41... Looking forward to upcoming projects! #neuroscience

041 0

Jascha Achterberg @achterbrain.bsky.social · 22/09/2023

Super interesting results on how economic conditions influence our social attitudes / motivation / cooperation behavior: "Older generations grew up with high growth and develop positive sum beliefs" Paper: ssrn.com/abstract=457... Reported in: www.ft.com/content/980c... #PsychSciSky #econsky 📈📉

Plot from FT article showing that people who grow up amidst weaker economic conditions tend to have more zero-sum mindsets and demotivating beliefs as adults.

0214 0

Jascha Achterberg @achterbrain.bsky.social · 19/09/2023

The MRC Cognition and Brain Sciences Unit at Cambridge started a new Master's program in Cognitive #Neuroscience ! If you know of a student looking for a great place to learn about the brain and its complex cognition, please point them in our direction! www.mrc-cbu.cam.ac.uk/study-with-u...

Screenshot of the webpage of the program

041 0

Jascha Achterberg @achterbrain.bsky.social · 10/09/2023

One well-kept secret of any Oxbridge college is that a rota is organised by students and academic staff for us all to take turns working on science in the public rooms of college on touristy summer days to provide an authentic scenery! 🧪 #AcademicSky #PhDSky

Picture of second court of St John’s College Cambridge on a sunny day, taken through a window.

051 0

Jascha Achterberg @achterbrain.bsky.social · 05/09/2023

How to choose a problem to work on? One of the super nice and short discussions of this, that I keep coming back to - for myself and also for helping other students - is: 'How To Choose a Good Scientific Problem' www.sciencedirect.com/science/arti... #PhDSky #AcademicSky 🧪 #neuroscience

How To Choose a Good Scientific Problem

Choosing good problems is essential for being a good scientist. But what is a good problem, and how do you choose one? The subject is not usually discussed explicitly within our profession. Scientists are expected to be smart enough to figure it out on their own and through the observation of their teachers. This lack of explicit discussion leaves a vacuum that can lead to approaches such as choosing problems that can give results that merit publication in valued journals, resulting in a job and tenure.

1115 0

Jascha Achterberg @achterbrain.bsky.social · 04/09/2023

How can brain-inspired learning mechanisms be best brought into artificial neural networks? Earlier this year I joined a great group of researchers from #neuroscience , AI / ML and engineering to address this question with a review! arxiv.org/abs/2305.112... Would be excited to hear your thoughts!

Figure 3 from the linked paper, showing different learning rules in a small neural network model.

1207 0

Jascha Achterberg @achterbrain.bsky.social · 01/09/2023

One of the posters I really liked at CCNeuro23 was presented by Marcus Gosh Multimodal units fuse-then-accumulate evidence across channels www.biorxiv.org/content/10.1... Check out their super nicely written preprint! Summary below. #neuroscience

Multimodal units fuse-then-accumulate evidence across channels

Key Points

We demonstrate that multimodal units are not necessary for accuracy or balancing speed/accuracy in classical multisensory tasks.

We introduce a novel set of tasks, based on comodulating the signals from multiple channels, in which multimodal units are critical.

We show that multimodal units enable networks to implement a fuse-then-accumulate algorithm, which excels in naturalistic settings, like predator-prey interactions, and is optimal for a wide class of multimodal problems.

Finally, we explore the link between single neurons properties and network behaviour.

040 0