iiinova — where your digital self keeps evolving

James Michaelov

@jamichaelov.bsky.social

4.3K followers 540 following 48 posts

Postdoc at Oxford. Research: language, the brain, NLP. jmichaelov.com

James Michaelov @jamichaelov.bsky.social · 28/09/2026

Looking forward to #SNL2026! I’ll be presenting the work published in our recent JML paper about language model scaling and the N400. Find me (or email/message) if you want to chat about predictive coding and NLP in the study of human language comprehension! Paper link: doi.org/10.1016/j.jm...

Title: Better language models better model the N400, but not reading time

Abstract: The probability of a word in context, as captured by large language models, is predictive of both behavioral and neural measures of human language processing. Intuitively, language models that are better at next-word prediction might better model predictability effects in human language comprehension. Yet recent work suggests that language models can become too good at next-word prediction to model reading time, implying that the aspects of human comprehension indexed by reading time do not track perfectly with predictability from language statistics alone. However, it is unknown whether this decoupling is true of reading time only, or whether it is intrinsic to online measures of comprehension more generally. To address this question, we turn to another robust and well-studied measure of online processing, the N400 component of the event-related brain potential. We compare how a language model’s size, number of training tokens, and performance on natural language benchmarks correlate with its ability to predict both reading time and N400 amplitude. Based on an analysis of 4 reading time datasets and 9 N400 datasets, we replicate past results for reading time, but find that larger language models, models that are trained on more data, and models that perform better at next-word prediction and other more complex natural language tasks are better able to predict N400 amplitude. We interpret this difference between the N400 and reading time measures as potentially revealing the comparative importance of semantic prediction in the neurocognitive processes indexed by the N400.

092

James Michaelov @jamichaelov.bsky.social · 11/06/2026

Here’s a summary of our main conclusions, and another link to the paper: arxiv.org/abs/2603.26539

000

James Michaelov @jamichaelov.bsky.social · 11/06/2026

Seems like a good time to share our new preprint about model openness! (with @catherinearnett.bsky.social @tylerachang.bsky.social Pamela D. Rivière, Samuel M. Taylor @camrobjones.bsky.social @seantrott.bsky.social @rplevy.bsky.social Ben Bergen, and Micah Altman): arxiv.org/abs/2603.26539

1193

James Michaelov @jamichaelov.bsky.social · 25/11/2025

Excited to announce that I’ll be presenting a paper at #NeurIPS this year! Reach out if you’re interested in chatting about LM training dynamics, architectural differences, shortcuts/heuristics, or anything at the CogSci/NLP/AI interface in general! #Neurips2025

2262

James Michaelov @jamichaelov.bsky.social · 12/06/2025

In the most extreme case, LMs assign sentences such as ‘the car was given a parking ticket by the explorer’ (unlikely but possible event) a lower probability than ‘the car was given a parking ticket by the brake’ (animacy-violating event, semantically-related final word) over half of the time. 2/3

110

James Michaelov @jamichaelov.bsky.social · 12/06/2025

New paper accepted at ACL Findings! TL;DR: While language models generally predict sentences describing possible events to have a higher probability than impossible (animacy-violating) ones, this is not robust for generally unlikely events and is impacted by semantic relatedness. 1/3

1213

James Michaelov @jamichaelov.bsky.social · 02/04/2024

But we also predict words that are RELATED to the word “full” such as “half” more than UNRELATED words like “mild”. Again, language models also show this effect:

100

James Michaelov @jamichaelov.bsky.social · 02/04/2024

But we are also more likely to predict a word RELATED to the described event (mountain biking) like “dirt” than an UNRELATED word like “table”, even though neither makes sense in context. We see the same effect in language models (a lower surprisal indicates a stronger prediction):

100

James Michaelov @jamichaelov.bsky.social · 10/12/2023

Looking forward to the final day of EMMLP! Let me know if you want to chat about our Findings paper: “Emergent inabilities? Inverse scaling over the course of pretraining” arxiv.org/abs/2305.14681 #EMNLP #EMNLP2023

Graph showing the performance at 12 tasks of the Pythia suite of models at various stages during training. Five tasks (TruthfulQA-MC1, TruthfulQA-MC2, Hindsight Neglect, Memo Trap, and Pattern Match Suppression) show clear evidence of inverse scaling, and three others (Redefine, Repetitive Algebra, and Resisting Correction) show inverse scaling for the largest model.

031