iiinova — where your digital self keeps evolving

Eva Dyer, Chandramouli Chandrasekaran, Nicholas A. Steinmetz, and Liam Paninski (ran out of characters!)

Han Yu (@hanyu42.bsky.social)

This work was led by @hanyu42.bsky.social who tirelessly worked to make this possible. In collaboration with Hanrui Lyu, Ethan Yixun Xu, @mostsquares.bsky.social, @kenjilee.bsky.social, Fan Yang, Andrew M. Shelton, Shawn Olsen, Sahar Minavi, Olivier Winter, @intlbrainlab.bsky.social, and

hanyu42.bsky.social

Ph.D. student at Columbia Center for Theoretical Neuroscience

We are still working on the codebase and aim to release a tool soon that users can download, fine-tune, and apply to their own datasets!

We evaluate NEMO on brain region localization by predicting the region of individual neurons (and nearby groups) using only the extracted features, and compare it to baseline methods. NEMO again outperforms both the VAE-based and supervised approaches.

100

We scale NEMO to the full IBL Brain-Wide Map dataset: 675 insertions from over 100 animals, yielding 37,017 high-quality neurons. Without using any labels, NEMO's features align closely with anatomical regions and are consistent across labs.

100

We benchmark NEMO against two SOTA cell-type classification methods, PhysMAP and a VAE (Beau et al., 2025), using two optotagged datasets from the mouse cerebellum and visual cortex. NEMO outperforms all baselines, including fully supervised models, with minimal fine-tuning.

100

We construct a paired dataset of spike trains and waveforms for all neurons, transforming spiking activity into an ACG image (Beau et al., 2025) that captures autocorrelation across firing rates. NEMO is trained to align ACGs and waveforms in a shared embedding space.

100

In vivo cell-type and brain region classification via multimodal...

Building on current multimodal cell-type classification methods (Lee et al. 2024 and Beau et al. 2025), we introduce a contrastive learning method for spiking activity and extracellular waveforms called NEMO. 🐟 Paper: Paper: openreview.net/forum?id=10J... Website: ibl-nemo.github.io

openreview.net

Current electrophysiological approaches can track the activity of many neurons, yet it is usually unknown which cell-types or brain areas are being recorded without further molecular or...

150

Matteo Carandini @carandinilab.net · 18/04/2025

Thrilled to share our state-of-the-art method for in vivo cell-type classification and brain region localization, NEMO, which is now now a spotlight at @iclr-conf.bsky.social ! We use NEMO to characterize the electrophysiological diversity of cell-types across the entire mouse brain. 🐭 🧪 🧠

16918

Reposted by Cole Hurwitz

Agreed! But here's a note of caution: in the brain, different behavioral contexts can engage completely different neurons! Julie Lee in our lab published this in 2022 (and I'm still digesting the implications). "Task specificity in mouse parietal cortex" www.cell.com/neuron/fullt...

doi.org

Redirecting

282

Cole Hurwitz @colehurwitz.bsky.social · 19/04/2025

Wow, this is fascinating. Thanks for sharing!

000

This point is really important. We will need to train models across diverse behavioral contexts, rather than over-interpreting results from a single experimental setup or task!

Totally agree. Data is the bottleneck in neuroscience, and far more costly than compute. But if we find that performance saturates with more data, it might reflect under-parameterized models. So both are key to interpret scaling.

060

@mehdiazabou.bsky.social and I have discussed this a lot and he has given me a lot of good feedback on this topic.

020

The Chinchilla paper (Training Compute-Optimal Large Language Models, Hoffmann et al., 2022) is widely regarded as a gold standard for empirically characterizing and optimizing scaling laws for large language models - arxiv.org/pdf/2203.15556

arxiv.org

However, we aren't close to a scaling law yet for neuroscience yet. Too many open questions! - What metric should we scale (encoding, decoding, forecasting)? - How many scaling laws do we need? Do we need separate scaling laws by region, species, or task? and many more questions 😀

210

Blake Richards @tyrellturing.bsky.social · 15/04/2025

Certainly, our work at IBL shows the benefits of scale (thanks for the shoutout @tyrellturing.bsky.social). However, scaling laws are most useful when defined by performance as a function of model size, data, and compute. The POYO paper comes closest, showing scaling with both model and data size.

230

Reposted by Cole Hurwitz

Very cool work here from @colehurwitz.bsky.social and others, and an important step towards a proper foundation model for neuroscience. Wonderful to see the scaling effects of moving from single session to multisession! (Also, a great use of the IBL data, IMO). #NeuroAI 🧠📈 🧪

0186

This amazing work was led by @yizi-zhang.bsky.social and @ppwang.bsky.social. In collaboration with @mehdiazabou.bsky.social, Alexandre Andre, Zixuan Wang, Hanrui Lyu, Eva Dyer, and Liam Paninski (and of course the International Brain Laboratory).

Major take aways from this project: (1) scale is important. We need to keep scaling up these approaches to new brain regions, tasks, and animals, (2) model comparison is hard! We need more community benchmarks for evaluation (like FALCON: snel-repo.github.io/falcon/).

Excitingly, NEDS’s learned embeddings exhibit emergent properties: even without explicit training, they are highly predictive of the brain regions in each recording.

100

In comparison to two state-of-the-art neural decoding models (POYO+ and NDT2), we show that NEDS achieves superior performance on predicting behavior from trial-aligned neural activity (we hyperparameter tune all models on a subset of 10 training animals).

By pretraining across 70+ animals, we see a large improvement in both encoding and decoding performance after fine-tuning on 10 held-out animals.

internationalbrainlab.com

We utilize trial-aligned data from the International Brain Laboratory repeated site dataset which has over 80 animals performing the same task. We predict the neural responses, wheel speed, whisker motion energy, choice, and "block" prior. www.internationalbrainlab.com/repro-ephys

Reproducible Electrophysiology — International Brain Laboratory

We tokenize neural activity, continuous, and discrete behaviors at each time step and then feed them into our shared transformer-based encoder model. We train NEDS by masking out and reconstructing neural activity, behavior, within-modality, and across modalities.

130

Current neurofoundation modeling approaches focus on either predicting neuronal responses (Wang et al. 2025) or behavior (POYO, POYO+, NDT2). Inspired by Schulz et al. 2025, we unify these two tasks using a multimodal masked modeling approach we call "Neural Encoding and Decoding at Scale (NEDS)"

190