Sign in

James Michaelov

@jamichaelov.bsky.social
4.3K followers 540 following 48 posts

Postdoc at Oxford. Research: language, the brain, NLP. jmichaelov.com

PostsRepliesMedia
James Michaelov @jamichaelov.bsky.social · 28/09/2026
Looking forward to #SNL2026! I’ll be presenting the work published in our recent JML paper about language model scaling and the N400. Find me (or email/message) if you want to chat about predictive coding and NLP in the study of human language comprehension! Paper link: doi.org/10.1016/j.jm...
Title: Better language models better model the N400, but not reading time

Abstract: The probability of a word in context, as captured by large language models, is predictive of both behavioral and neural measures of human language processing. Intuitively, language models that are better at next-word prediction might better model predictability effects in human language comprehension. Yet recent work suggests that language models can become too good at next-word prediction to model reading time, implying that the aspects of human comprehension indexed by reading time do not track perfectly with predictability from language statistics alone. However, it is unknown whether this decoupling is true of reading time only, or whether it is intrinsic to online measures of comprehension more generally. To address this question, we turn to another robust and well-studied measure of online processing, the N400 component of the event-related brain potential. We compare how a language model’s size, number of training tokens, and performance on natural language benchmarks correlate with its ability to predict both reading time and N400 amplitude. Based on an analysis of 4 reading time datasets and 9 N400 datasets, we replicate past results for reading time, but find that larger language models, models that are trained on more data, and models that perform better at next-word prediction and other more complex natural language tasks are better able to predict N400 amplitude. We interpret this difference between the N400 and reading time measures as potentially revealing the comparative importance of semantic prediction in the neurocognitive processes indexed by the N400.
092
Reposted by James Michaelov
Sean Trott @seantrott.bsky.social · 31/08/2026
New paper out in @openmindjournal.bsky.social! "Large Language Models as Distributional Baselines for Language Tasks". With @jamichaelov.bsky.social , @camrobjones.bsky.social , Tyler Chang, and Ben Bergen. direct.mit.edu/opmi/article...
direct.mit.edu
Large Language Models as Distributional Baselines for Language Tasks
Abstract. In order to ask questions about the mechanisms underpinning human cognition, researchers must control for properties of stimuli that could confound detected effects. In experiments involving...
1126
James Michaelov @jamichaelov.bsky.social · 01/07/2026
I’ll be in San Diego for #ACL2026 and presenting this work at @conll-conf.bsky.social #CoNLL2026 on July 3rd! Feel free to reach out if you want to chat!
0110
James Michaelov @jamichaelov.bsky.social · 11/06/2026
Seems like a good time to share our new preprint about model openness! (with @catherinearnett.bsky.social @tylerachang.bsky.social Pamela D. Rivière, Samuel M. Taylor @camrobjones.bsky.social @seantrott.bsky.social @rplevy.bsky.social Ben Bergen, and Micah Altman): arxiv.org/abs/2603.26539
1193
James Michaelov @jamichaelov.bsky.social · 27/03/2026
Had a great first day at #HSP2026 yesterday! Looking forward to presenting on the relationship between reading time, n-grams, and language model scaling at the 12.10-2pm poster session today!
040
James Michaelov @jamichaelov.bsky.social · 04/12/2025
Presenting this at the poster session this morning (11-2pm) at #5109
020
James Michaelov @jamichaelov.bsky.social · 01/12/2025
Looking forward to #NeurIPS25 this week 🏝️! I'll be presenting at Poster Session 3 (11-2 on Thursday). Feel free to reach out!
0103
James Michaelov @jamichaelov.bsky.social · 25/11/2025
Excited to announce that I’ll be presenting a paper at #NeurIPS this year! Reach out if you’re interested in chatting about LM training dynamics, architectural differences, shortcuts/heuristics, or anything at the CogSci/NLP/AI interface in general! #Neurips2025
2262
Reposted by James Michaelov
Catherine Arnett @catherinearnett.bsky.social · 27/07/2025
I’m in Vienna all week for @aclmeeting.bsky.social and I’ll be presenting this paper on Wednesday at 11am (Poster Session 4 in HALL X4 X5)! Reach out if you want to chat about multilingual NLP, tokenizers, and open models!
0171
James Michaelov @jamichaelov.bsky.social · 12/06/2025
New paper accepted at ACL Findings! TL;DR: While language models generally predict sentences describing possible events to have a higher probability than impossible (animacy-violating) ones, this is not robust for generally unlikely events and is impacted by semantic relatedness. 1/3
1213
Reposted by James Michaelov
Catherine Arnett @catherinearnett.bsky.social · 05/06/2025
My paper with @tylerachang.bsky.social and @jamichaelov.bsky.social will appear at #ACL2025NLP! The updated preprint is available on arxiv. I look forward to chatting about bilingual models in Vienna!
182
Reposted by James Michaelov
Catherine Arnett @catherinearnett.bsky.social · 07/03/2025
✨New pre-print✨ Crosslingual transfer allows models to leverage their representations for one language to improve performance on another language. We characterize the acquisition of shared representations in order to better understand how and when crosslingual transfer happens.
2367
Reposted by James Michaelov
James Michaelov @jamichaelov.bsky.social · 10/11/2024
With all the new people here on Bluesky, I think it’s a good time to (re-)introduce myself. I’m a postdoc at MIT carrying out research at the intersection of the cognitive science of language and AI. Here are some of the things I’ve worked on in the last year 🧵:
1242
James Michaelov @jamichaelov.bsky.social · 11/11/2024
Excited to be at #EMNLP #EMNLP2024 this year! Especially interested in chatting about the intersection of cognitive science/psycholinguistics and AI/NLP, training dynamics, robustness/reliability, meaning, and evaluation
1110
James Michaelov @jamichaelov.bsky.social · 10/11/2024
With all the new people here on Bluesky, I think it’s a good time to (re-)introduce myself. I’m a postdoc at MIT carrying out research at the intersection of the cognitive science of language and AI. Here are some of the things I’ve worked on in the last year 🧵:
1242
James Michaelov @jamichaelov.bsky.social · 02/04/2024
In the interest of actually posting about my research on here: We know that the predictions that language models make are similar to those that humans make as we process language, but how similar? aclanthology.org/2022.conll-1... 🧵:
100
James Michaelov @jamichaelov.bsky.social · 10/12/2023
Looking forward to the final day of EMMLP! Let me know if you want to chat about our Findings paper: “Emergent inabilities? Inverse scaling over the course of pretraining” arxiv.org/abs/2305.14681 #EMNLP #EMNLP2023
Graph showing the performance at 12 tasks of the Pythia suite of models at various stages during training. Five tasks (TruthfulQA-MC1, TruthfulQA-MC2, Hindsight Neglect, Memo Trap, and Pattern Match Suppression) show clear evidence of inverse scaling, and three others (Redefine, Repetitive Algebra, and Resisting Correction) show inverse scaling for the largest model.
031
James Michaelov @jamichaelov.bsky.social · 08/12/2023
Presenting this at the 2pm poster session today! #EMNLP #EMNLP2023
020
James Michaelov @jamichaelov.bsky.social · 01/12/2023
Excited to present our paper "Structural Priming Demonstrates Abstract Grammatical Representations in Multilingual Language Models" at EMNLP next week! arxiv.org/abs/2311.09194
arxiv.org
Structural Priming Demonstrates Abstract Grammatical...
Abstract grammatical knowledge - of parts of speech and grammatical patterns - is key to the capacity for linguistic generalization in humans. But how abstract is grammatical knowledge in large...
060