Sign in

Jaap Jumelet

@jumelet.bsky.social
758 followers 291 following 38 posts

Postdoc @rug.nl with Arianna Bisazza. Interested in NLP, interpretability, syntax, language acquisition and typology.

PostsRepliesMedia
Reposted by Jaap Jumelet
Multilingual Representation Workshop @ EMNLP 2026 @mrl-workshop.bsky.social · 20/07/2026
🎤 Speaker lineup drop! The MRL 2026 Workshop is bringing academia and industry together: @afaji.bsky.social , MBZUAI @mdlhx.bsky.social , KU Leuven @jumelet.bsky.social , Uni of Groningen Ahmet Üstün, Cohere See you all this October in Budapest! #EMNLP #MRL2026
192
Reposted by Jaap Jumelet
Francesca Padovani @frap98.bsky.social · 14/04/2026
I’m very happy to share that my latest paper on the 𝐚𝐜𝐪𝐮𝐢𝐬𝐢𝐭𝐢𝐨𝐧 𝐨𝐟 𝐯𝐞𝐫𝐛 𝐦𝐞𝐚𝐧𝐢𝐧𝐠, tested on models trained under CDL data vs ADL data , has been accepted for an 𝐨𝐫𝐚𝐥 𝐩𝐫𝐞𝐬𝐞𝐧𝐭𝐚𝐭𝐢𝐨𝐧 at the upcoming edition of 𝐂𝐨𝐠𝐒𝐜𝐢, which will take place at the end of July in Rio de Janeiro.💃
1246
Reposted by Jaap Jumelet
Vitalii Hirak @v-hirak.bsky.social · 08/02/2026
Our paper has been accepted to #EACL2026 main conference! Together with @jumelet.bsky.social and @arianna-bis.bsky.social, we study the effect of target language typology on the difficulty of state-of-the-art neural machine translation. arXiv preprint: arxiv.org/abs/2602.03551 1/6 Our findings ⬇️
arxiv.org
Assessing the Impact of Typological Features on Multilingual Machine Translation in the Age of Large Language Models
Despite major advances in multilingual modeling, large quality disparities persist across languages. Besides the obvious impact of uneven training resources, typological properties have also been prop...
1151
Reposted by Jaap Jumelet
GroNLP @gronlp.bsky.social · 18/12/2025
👀 Look what 🎅 has broght just before Christmas 🎁: a brand new Research Master in Natural Language Processing at @facultyofartsug.bsky.social @rug.nl Program: www.rug.nl/masters/natu... Applications (2026/2027) are open! Come and study with us (you will also learn why we have a 🐮 in our logo)
rug.nl
Natural Language Processing
How do you build Large Language Models? How do humans experience Natural Language Processing (NLP) applications in their daily lives? And how can we...
02515
Reposted by Jaap Jumelet
Leonie Weissweiler @weissweiler.bsky.social · 11/12/2025
🧑‍🔬I’m recruiting PhD students in Natural Language Processing @unileipzig.bsky.social Computer Science, together with @scadsai.bsky.social! Topics include, but aren’t limited to: 🔎Linguistic Interpretability 🌍Multilingual Evaluation 📖Computational Typology Please share! #NLProc #NLP
14225
Reposted by Jaap Jumelet
Leonie Weissweiler @weissweiler.bsky.social · 19/11/2025
📢Out now in NEJLT!📢 In each of these sentences, a verb that doesn't usually encode motion is being used to convey that an object is moving to a destination. Given that these usages are rare, complex, and creative, we ask: Do LLMs understand what's going on in them? 🧵1/7
2153
Reposted by Jaap Jumelet
Jennifer Hu @jennhu.bsky.social · 10/11/2025
New work to appear @ TACL! Language models (LMs) are remarkably good at generating novel well-formed sentences, leading to claims that they have mastered grammar. Yet they often assign higher probability to ungrammatical strings than to grammatical strings. How can both things be true? 🧵👇
Screenshot of a figure with two panels, labeled (a) and (b). The caption reads: "Figure 1: (a) Illustration of messages (left) and strings (right) in toy domain. Blue = grammatical strings. Red = ungrammatical strings. (b) Surprisal (negative log probability) assigned to toy strings by GPT-2."
29220
Jaap Jumelet @jumelet.bsky.social · 06/11/2025
I'm in Suzhou to present our work on MultiBLiMP, Friday @ 11:45 in the Multilinguality session (A301)! Come check it out if your interested in multilingual linguistic evaluation of LLMs (there will be parse trees on the slides! There's still use for syntactic structure!) arxiv.org/abs/2504.02768
0277
Reposted by Jaap Jumelet
GroNLP @gronlp.bsky.social · 27/10/2025
With only a week left for #EMNLP2025, we are happy to announce all the works we 🐮 will present 🥳 - come and say "hi" to our posters and presentations during the Main and the co-located events (*SEM and workshops) See you in Suzhou ✈️
accepted papers at main conference and findingsaccepted papers at TACL and workshops
0165
Jaap Jumelet @jumelet.bsky.social · 15/10/2025
🌍Introducing BabyBabelLM: A Multilingual Benchmark of Developmentally Plausible Training Data! LLMs learn from vastly more data than humans ever experience. BabyLM challenges this paradigm by focusing on developmentally plausible data We extend this effort to 45 new languages!
14416
Jaap Jumelet @jumelet.bsky.social · 01/08/2025
Happening now at the SIGTYP poster session! Come talk to Leonie and me about MultiBLiMP!
1202
Reposted by Jaap Jumelet
Jelle Zuidema 🟥 @wzuidema.bsky.social · 29/07/2025
I'll be in Vienna only from tomorrow, but today my star PhD student Marianne is already presenting some of our work: BLIMP-NL, in which we create a large new dataset for syntactic evaluation of Dutch LLMs, and learn a lot about dataset creation, LLM evaluation and grammatical abilities on the way.
1111
Reposted by Jaap Jumelet
Arianna Bisazza @arianna-bis.bsky.social · 19/06/2025
Proud to introduce TurBLiMP, the 1st benchmark of minimal pairs for free-order, morphologically rich Turkish language! Pre-print: arxiv.org/abs/2506.13487 Fruit of an almost year-long project by amazing MS student @ezgibasar.bsky.social in collab w/ @frap98.bsky.social and @jumelet.bsky.social
arxiv.org
TurBLiMP: A Turkish Benchmark of Linguistic Minimal Pairs
We introduce TurBLiMP, the first Turkish benchmark of linguistic minimal pairs, designed to evaluate the linguistic abilities of monolingual and multilingual language models (LMs). Covering 16 linguis...
1112
Reposted by Jaap Jumelet
Casper Albers 🟥 @casperalbers.nl · 13/06/2025
Ik snap niet dat hier niet meer ophef over is: Het binnenhalen van Amerikaanse wetenschappers wordt betaalt door Nederlandse academici geen inflatiecorrectie op hun salaris te geven. 1/2
56535
Reposted by Jaap Jumelet
Catherine Arnett @catherinearnett.bsky.social · 05/06/2025
My paper with @tylerachang.bsky.social and @jamichaelov.bsky.social will appear at #ACL2025NLP! The updated preprint is available on arxiv. I look forward to chatting about bilingual models in Vienna!
182
Reposted by Jaap Jumelet
Francesca Padovani @frap98.bsky.social · 30/05/2025
“Child-Directed Language Does Not Consistently Boost Syntax Learning in Language Models” I’m happy to share that the preprint of my first PhD project is now online! 🎊 Paper: arxiv.org/abs/2505.23689
arxiv.org
Child-Directed Language Does Not Consistently Boost Syntax Learning in Language Models
Seminal work by Huebner et al. (2021) showed that language models (LMs) trained on English Child-Directed Language (CDL) can reach similar syntactic abilities as LMs trained on much larger amounts of ...
26117
Reposted by Jaap Jumelet
Neil Renic @ncrenic.bsky.social · 28/05/2025
"A well-delivered lecture isn’t primarily a delivery system for information. It is an ignition point for curiosity, all the better for being experienced in an audience." Marvellous defence of the increasingly maligned university experience by @patporter76.bsky.social thecritic.co.uk/university-a...
thecritic.co.uk
University: a good idea | Patrick Porter | The Critic Magazine
A former student of mine has penned an attack on universities, derived from their own disappointing experience studying Politics and International Relations at the place where I ply my trade. In short...
05819
Reposted by Jaap Jumelet
Miryam de Lhoneux @mdlhx.bsky.social · 16/05/2025
Interested in multilingual tokenization in #NLP? Lisa Beinborn and I are hiring! PhD candidate position in Göttingen, Germany: www.uni-goettingen.de/de/644546.ht... PostDoc position in Leuven, Belgium: www.kuleuven.be/personeel/jo... Deadline 6th of June
uni-goettingen.de
Stellen OBP - Georg-August-Universität Göttingen
Webseiten der Georg-August-Universität Göttingen
22513
Reposted by Jaap Jumelet
BlackboxNLP @blackboxnlp.bsky.social · 15/05/2025
BlackboxNLP, the leading workshop on interpretability and analysis of language models, will be co-located with EMNLP 2025 in Suzhou this November! 📆 This edition will feature a new shared task on circuits/causal variable localization in LMs, details here: blackboxnlp.github.io/2025/task
3218
Reposted by Jaap Jumelet
Leshem (Legend) Choshen @EMNLP @lchoshen.bsky.social · 09/05/2025
Close your books, test time! The evaluation pipelines are out, baselines are released & the challenge is on There is still time to join and We are excited to learn from you on pretraining and human-model gaps *Don't forget to fastEval on checkpoints github.com/babylm/evalu... 📈🤖🧠 #AI #LLMS
0104
Reposted by Jaap Jumelet
Seth Aycock @sethjsa.bsky.social · 25/04/2025
Pleased to announce our paper was accepted at ICLR 2025 as a Spotlight! I will present our poster on Saturday April 26, 3-5pm, Poster #241. Hope to see you there! arxiv.org/abs/2409.19151
arxiv.org
Can LLMs Really Learn to Translate a Low-Resource Language from One Grammar Book?
Extremely low-resource (XLR) languages lack substantial corpora for training NLP models, motivating the use of all available resources such as dictionaries and grammar books. Machine Translation from ...
0173
Reposted by Jaap Jumelet
Jirui Qi @jiruiqi.bsky.social · 11/04/2025
✨ New Paper ✨ [1/] Retrieving passages from many languages can boost retrieval augmented generation (RAG) performance, but how good are LLMs at dealing with multilingual contexts in the prompt? 📄 Check it out: arxiv.org/abs/2504.00597 (w/ @arianna-bis.bsky.social @Raquel_Fernández) #NLProc
145
Reposted by Jaap Jumelet
Jaap Jumelet @jumelet.bsky.social · 07/04/2025
✨New paper ✨ Introducing 🌍MultiBLiMP 1.0: A Massively Multilingual Benchmark of Minimal Pairs for Subject-Verb Agreement, covering 101 languages! We present over 125,000 minimal pairs and evaluate 17 LLMs, finding that support is still lacking for many languages. 🧵⬇️
37622
Reposted by Jaap Jumelet
Arianna Bisazza @arianna-bis.bsky.social · 08/04/2025
Modern LLMs "speak" hundreds of languages... but do they really? Multilinguality claims are often based on downstream tasks like QA & MT, while *formal* linguistic competence remains hard to gauge in lots of languages Meet MultiBLiMP! (joint work w/ @jumelet.bsky.social & @weissweiler.bsky.social)
2216
Jaap Jumelet @jumelet.bsky.social · 07/04/2025
✨New paper ✨ Introducing 🌍MultiBLiMP 1.0: A Massively Multilingual Benchmark of Minimal Pairs for Subject-Verb Agreement, covering 101 languages! We present over 125,000 minimal pairs and evaluate 17 LLMs, finding that support is still lacking for many languages. 🧵⬇️
37622
Reposted by Jaap Jumelet
Qing Yao @qyao.bsky.social · 31/03/2025
LMs learn argument-based preferences for dative constructions (preferring recipient first when it’s shorter), consistent with humans. Is this from memorizing preferences in training? New paper w/ @kanishka.bsky.social , @weissweiler.bsky.social , @kmahowald.bsky.social arxiv.org/abs/2503.20850
examples from direct and prepositional object datives with short-first and long-first word orders: 
DO (long first): She gave the boy who signed up for class and was excited it.
PO (short first): She gave it to the boy who signed up for class and was excited.
DO (short first): She gave him the book that everyone was excited to read.
PO (long-first): She gave the book that everyone was excited to read to him.
1177
Reposted by Jaap Jumelet
Siyuan Song @siyuansong.bsky.social · 12/03/2025
New preprint w/ @jennhu.bsky.social @kmahowald.bsky.social : Can LLMs introspect about their knowledge of language? Across models and domains, we did not find evidence that LLMs have privileged access to their own predictions. 🧵(1/8)
26116
Reposted by Jaap Jumelet
Arianna Bisazza @arianna-bis.bsky.social · 01/03/2025
Excited to be traveling to Estonia for the 1st time to give a keynote @nodalida.bsky.social. I'll talk about using NNs to study language evolution & acquisition. A teaser: It won't be about LLMs 🙃 Also I've just moved from X, so this was my very first post... Pls help out by connecting with me!
4599
Reposted by Jaap Jumelet
Leonie Weissweiler @weissweiler.bsky.social · 20/02/2025
✨New paper✨ Linguistic evaluations of LLMs often implicitly assume that language is generated by symbolic rules. In a new position paper, @adelegoldberg.bsky.social, @kmahowald.bsky.social and I argue that languages are not Lego sets, and evaluations should reflect this! arxiv.org/pdf/2502.13195
16819
Reposted by Jaap Jumelet
Michael Hanna @michaelwhanna.bsky.social · 19/12/2024
Sentences are partially understood before they're fully read. How do LMs incrementally interpret their inputs? In a new paper, @amuuueller.bsky.social and I use mech interp tools to study how LMs process structurally ambiguous sentences. We show LMs rely on both syntactic & spurious features! 1/10
17115
Reposted by Jaap Jumelet
Luke Rendell @lrendell.bsky.social · 15/12/2024
Counterpoint: theconversation.com/are-we-reall...
theconversation.com
Are we really about to talk to whales?
It’s certainly an exciting time to study communication in whales and dolphins.
131
Reposted by Jaap Jumelet
AmsterdamNLP @amsterdamnlp.bsky.social · 15/12/2024
Jaap Jumelet @jumelet.bsky.social successfully defended his dissertation this week, closing of 6 productive years as master and PhD student at AmsterdamNLP. bsky.app/profile/jume... Here's a good thread on some of the highlights of his work! bsky.app/profile/jume... All the best at @ groningennlp
0141
Jaap Jumelet @jumelet.bsky.social · 11/12/2024
I have succesfully defended my thesis yesterday! Huge thank you to my supervisors @wzuidema.bsky.social and Raquel Fernandez for their guidance, my paranymphs Emile & Jop for their support, my defense committee for their questions, and my brother for photobombing this official portrait!
9470
Reposted by Jaap Jumelet
Shane Steinert-Threlkeld @shane.st · 02/12/2024
Excited that the final version of our paper on filtered corpus training is out now in TACL! (Original thread is on the bad site, but the paper's title is a succinct summary.) doi.org/10.1162/tacl... Always a pleasure to work with Abhinav Patil and @jumelet.bsky.social , who deserve the real credit
doi.org
Filtered Corpus Training (FiCT) Shows that Language Models Can Generalize from Indirect Evidence
Abstract. This paper introduces Filtered Corpus Training, a method that trains language models (LMs) on corpora with certain linguistic constructions filtered out from the training data, and uses it t...
1173
Jaap Jumelet @jumelet.bsky.social · 29/11/2024
My PhD thesis is now available online! It is called "Finding Structure in Language Models" and presents the work I have done in the past 4 years on the intersection of NLP, interpretability, and linguistics. I'll defend it on December 10, streamed here: jumelet.ai/phd arxiv.org/abs/2411.16433 🧵
811414
Reposted by Jaap Jumelet
Gabriele Sarti @gsarti.com · 21/11/2024
Our lab @gronlp.bsky.social at the University of Groningen has joined the chat! 🦋
0273