iiinova — where your digital self keeps evolving

Later in the paper, we walk through some examples of empirical work in speech and language neuroscience where LLMs might provide some insights 🧠

131 0

Principle 2: They replace rule-based learning with a generic, self-supervised, context-driven learning algorithm for reproducing the statistical structure of real-world language.

131 0

Principle 1: They encode all of the structures of natural language (syntax, semantics, pragmatics, etc.) in the geometry of a continuous, high-dimensional embedding space—i.e., a neural population code.

121 0

Sam Nastase @samnastase.bsky.social · 01/10/2025

New perspective piece out in @cp-neuron.bsky.social with Zaid Zada, @adelegoldberg.bsky.social, and Uri Hasson! We try to articulate some of our excitement about LLMs and discuss what kinds of insights they might provide into the neural computations supporting natural language in the human brain.

18725 0

I'm recruiting PhD students to join my new lab in Fall 2026! The Shared Minds Lab at @usc.edu will combine deep learning and ecological human neuroscience to better understand how we communicate our thoughts from one brain to another.

912474 0

Finally, we developed a set of interactive tutorials for preprocessing and running encoding models to get you started. Happy to hear any feedback or field any questions about the dataset! hassonlab.github.io/podcast-ecog...

072 0

We validated both the data and stimulus features using encoding models, replicating previous findings showing an advantage for LLM embeddings.

We also provide word-level transcripts and stimulus features ranging from low-level acoustic features to large language model embeddings.

100 0

We recorded ECoG data in nine subjects while they listened to a 30-minute story. We provide a minimally preprocessed derivative of the raw data, ready to be used.

Check out Zaid's open "Podcast" ECoG dataset for natural language comprehension (w/ Hasson Lab). The paper is now out at Scientific Data (nature.com/articles/s41...) and the data are available on OpenNeuro (openneuro.org/datasets/ds0...).

13916 0

We then tested the extent to which each of these 58 languages can predict the brain activity of our participants. We found that languages that are more similar to the listener’s native language, the better the prediction:

What about multilingual models? We translated the story from English to 57 other languages spanning 14 families, and extracted embeddings for each from multilingual BERT. We visualized the dissimilarity matrix using MDS and found clusters corresponding to language family types.

We found that models trained to predict neural activity for one language generalize to different subjects listening to the same content in a different language, across high-level language and default-mode regions.

We extracted embeddings from three unilingual BERT models—trained on entirely different languages)—and found that (with a rotation) they converge onto similar embeddings, especially in the middle layers:

100 0

We used naturalistic fMRI and language models (LMs) to identify neural representations of the shared conceptual meaning of the same story as heard by native speakers of three languages: English, Chinese, and French.

100 0

We hypothesized that the brains of native speakers of different languages would converge on the same supra-linguistic conceptual structures when listening to the same story in their respective languages:

100 0

Previous research has found that language models trained on different languages learn embedding spaces with similar geometry. This suggests that internal geometry of different languages may converge on similar conceptual structures:

100 0

How do different languages converge on a shared neural substrate for conceptual meaning? Happy to share a new preprint led by Zaid Zada that specifically addresses this question:

1355 0

Taking a slightly different approach, we assess how well specific model features capture larger-scale patterns of connectivity. We find that feature-specific model connectivity partly recapitulates stimulus-driven cortical network configuration.

100 0

We observe a clear progression of feature-specific connectivity from early auditory to lateral temporal areas, advancing from acoustic-driven connectivity to speech- and finally language-driven connectivity.

100 0

We show that early auditory areas are coupled to intermediate language areas via lower-level acoustic and speech features. In contrast, higher-order language and default-mode regions are predominantly coupled through more abstract language features.

100 0

We developed a model-based framework for quantifying stimulus-driven, feature-specific connectivity between regions. We used parcel-wise encoding models to align feature-specific embeddings to brain activity and then evaluated how well these models generalize to other parcels.

100 0

Following the logic of intersubject correlation (ISC) analysis, intersubject functional connectivity (ISFC) isolates stimulus-driven connectivity between regions (e.g., in response to naturalistic stimuli)—but is agnostic to the content of the stimulus shared between regions.

100 0

We used fMRI data collected while subjects listened to naturalistic spoken stories and extracted three types of linguistic embeddings for the same stimuli from the Whisper speech and language model. We call these "acoustic", "speech", and "language" embeddings.

100 0