iiinova — where your digital self keeps evolving

Joel Ye

@joelye.bsky.social

NeuroAI PhD student @ Carnegie Mellon. Being a brain-computer interface.

Joel Ye @joelye.bsky.social · 07/02/2025

NDT3 is brought to you by a huge team of data contributors from @corticalbionics.bsky.social @raeedcho.bsky.social @jenpitt.bsky.social . ? (Apologies for the shortlist, not wired into the bsky community yet)

110

Joel Ye @joelye.bsky.social · 07/02/2025

Link: doi.org/10.1101/2025... Code: github.com/joel99/ndt3 . Weights are posted on HuggingFace, includes a notebook for fine-tuning NDT3 on a new dataset.

100

Joel Ye @joelye.bsky.social · 07/02/2025

Lowlights: - NDT3 scales weakly, likely due to cross-subject transfer issues (but there's some rzn to believe cross-attention encoders, like in POYO, wouldn't fare better) - NDT3 will still overfit some types of stereotypy in neural data, e.g. reach angle. Post-training likely needed to resolve

110

Joel Ye @joelye.bsky.social · 07/02/2025

Highlights: - Pretrained on 2000 hours. By data/params, NDT3 brings motor BCI to the BERT era! - Evaluated on 8 datasets, (including 3 from FALCON, go beat NDT3 snel.ai/falcon !) - Gains ~generalize to distribution shifts in neural data -- e.g. in time, posture

100

Joel Ye @joelye.bsky.social · 07/02/2025

What it is: - NDT3 is "just" an autoregressive Transformer, for decoding behavior from neural spiking activity - We tokenize behavior timeseries by splitting and neural data by patching (a la NDT2/ViT). Thus, the model can ingest most motor brain-computer interface datasets

100

Joel Ye @joelye.bsky.social · 07/02/2025

Neural Data Transformer 3 is preprinted! We think it can be an off-the-shelf model for all your intracortical motor decoding needs. Decoding gains over your multisession model expected if you have <1.5 hours of data.

2184