Sign in

Martin Tutek

@mtutek.bsky.social
373 followers 403 following 95 posts

Postdoc @ TakeLab, University of Zagreb | Working on interpretability & safety of LLMs. mttk.github.io

PostsRepliesMedia
Martin Tutek @mtutek.bsky.social · 09/09/2026
Take a break from reading up on Navier-Stokes drama and add October 29th to your calendar, as we have a stellar lineup of speakers at this years' BlackboxNLP!
051
Reposted by Martin Tutek
BlackboxNLP @blackboxnlp.bsky.social · 31/08/2026
Due to the unprecedented amount of submissions to BlackboxNLP 2026, we will announce the decisions for Main, Special and ARR commitments on September 2 AoE. Thanks for your patience!
063
Martin Tutek @mtutek.bsky.social · 22/08/2026
The @blackboxnlp.bsky.social reproducibility track is in an urgent need for reviewers! If you can review a paper (or two), these bite-sized papers deserve some attention, and reviewing them will leave you fulfilled! 🥰 Please reach out if you can help out! RTs appreciated 👐
035
Reposted by Martin Tutek
Debora Nozza @deboranozza.bsky.social · 07/08/2026
Very happy that #IC2S22027 will be in Milan at Università Bocconi! I’m really looking forward to welcoming the computational social science community to my university and to Milan, and excited to be part of the team organizing it. See you in 2027! 🇮🇹✨
15212
Reposted by Martin Tutek
BlackboxNLP @blackboxnlp.bsky.social · 21/07/2026
With a high volume of submissions this year, we're recruiting additional reviewers for BlackboxNLP 2026! 🗓 Review deadline: August 17 (AoE) 🗓 Review load: 2-4 papers 📝 Sign up: forms.gle/tha166UQZYqX...
065
Reposted by Martin Tutek
MilaNLP Lab @milanlp.bsky.social · 16/07/2026
For today's reading group, @veraneplenbroek.bsky.social presented "Old Habits Die Hard: How Conversational History Geometrically Traps LLMs" by Simhi et al. (2026) Paper: arxiv.org/abs/2603.03308 #NLProc
0125
Reposted by Martin Tutek
BlackboxNLP @blackboxnlp.bsky.social · 16/07/2026
⏳ The BlackboxNLP 2026 Reproducibility Challenge deadline has been extended to July 24 (AoE) ⏳ If you've been working on a robustness check, ablation, or replication of recent NLP interpretability work, you have a bit more time to get your submission in.
186
Reposted by Martin Tutek
BlackboxNLP @blackboxnlp.bsky.social · 11/07/2026
⏳ One week left! The submission deadline for BlackboxNLP 2026 is July 17 (AoE). If you're working on analyzing or interpreting neural networks for NLP, now's the time to get your paper in. 📍 Co-located with #EMNLP2026 in Budapest 🔗 blackboxnlp.github.io
062
Reposted by Martin Tutek
BlackboxNLP @blackboxnlp.bsky.social · 01/07/2026
🏆 Announcing the NDIF Best Paper Award for the BlackboxNLP 2026 Reproducibility Challenge! @ndif-team.bsky.social Reproduce an interp. finding with nnsight + NDIF, open-source it, and push it further. Winners will receive a $500 prize, and will be invited to present at the workshop!
183
Reposted by Martin Tutek
takelab.bsky.social @takelab.bsky.social · 30/06/2026
At ICML '26, TakeLab members are presenting three papers. Make sure to stop by if interested in language model interpretability, coding agent evaluations, or pluralistic alignment. Details below 👇🧵 @icmlconf.bsky.social #ICML2026 🇰🇷
141
Reposted by Martin Tutek
EACL 2027 @eaclmeeting.bsky.social · 30/06/2026
📣 #EACL2027 updates: the Call for Papers is live, and our keynote speakers are confirmed! Main conference: 9–14 March 2027. Special theme: The Human in Language. 🧵👇 2027.eacl.org/calls/papers/
2027.eacl.org
Call for Papers
Official website for the 2027 Conference of the European Chapter of the Association for Computational Linguistics
12113
Reposted by Martin Tutek
Naomi Saphra @nsaphra.bsky.social · 15/06/2026
We don’t always know what problems are hard for LLMs. So devs evaluate on tasks HUMANS find hard or on broad benchmarks. What if we could instead anticipate which scenarios a model will fail on—all without evaluating specific input examples? 🧵NEW PAPER by @jenniferlumeng.bsky.social
313734
Reposted by Martin Tutek
Naomi Saphra @nsaphra.bsky.social · 09/06/2026
✨ it's coming ✨ NEMI 2026 will be lit. It will also be the new BU interp supergroup's debut ball. Come meet us!
nemiconf.github.io
The 3rd New England Mechanistic Interpretability (NEMI) Workshop
1274
Martin Tutek @mtutek.bsky.social · 02/06/2026
With the large influx of submissions and a faster pace of research, reproducibility is more important than ever. With this reproducibility challenge, we want to put the focus on best practices wrt. baselines🧱, ablations🌈, eval🔎 and generalizability🗺️ of interpretability!
073
Reposted by Martin Tutek
BlackboxNLP @blackboxnlp.bsky.social · 28/05/2026
📣 Announcing the BlackboxNLP 2026 Reproducibility Challenge! A new track dedicated to rigorous robustness checks of NLP interpretability work - stress-testing baselines, ablations, generalizability, and evaluation.
1165
Reposted by Martin Tutek
Markus Eichhorn @markuseichhorn.bsky.social · 07/05/2026
Bingo! Do I win a prize now?
Comic strip titles self-sabotaging academic habits that feel normal, featuring nine panels.
1122583
Reposted by Martin Tutek
Itay Itzhak @ COLM 🍁 @itay-itzhak.bsky.social · 23/04/2026
Ever used a top-ranked LLM that just... felt wrong for you? You’re not alone. Instead of leaderboards, many of us turn to "vibe-testing" - manually comparing models to our own needs. But can we turn these feelings into a structured evaluation? New paper: "From Feelings to Metrics" 🧵
271
Reposted by Martin Tutek
MilaNLP Lab @milanlp.bsky.social · 21/04/2026
We are delighted to welcome @marlutz.bsky.social to our lab over the next few months! 🎉 She'll work on the representation of different demographic groups in LLMs. #NLProc
0248
Reposted by Martin Tutek
Maria Antoniak @mariaa.bsky.social · 21/04/2026
FYI #ACL2026 has an unusual registration system this year, and probably a lot of people who want to attend will not be able to. Spots are limited to 3.5k people, and only presenting authors can register during the first phase. Then, *if* there are spots left, others can try to register.
2026.aclweb.org
Registration
Official website for the 64th Annual Meeting of the Association for Computational Linguistics
2113
Reposted by Martin Tutek
Anna Rogers @annarogers.bsky.social · 13/04/2026
📢 The workshop on Insights from negative results will be back at EMNLP'26! Your most-insightful failures can be submitted in 4 pages by June 25. It's also possible to commit short papers reviewed through ARR. insights-workshop.github.io/2026/cfp
insights-workshop.github.io
2026 Call for Papers
Workshop on Insights from Negative Results in NLP
03211
Reposted by Martin Tutek
Abhilasha Ravichander @lasha.bsky.social · 12/04/2026
How can generative AI better support human creativity, without limiting it? If you have thoughts, we invite submissions to our ICML workshop on Generative AI, Creativity, and Human-AI Co-Creation 📍 July 2026, Seoul 📄 Submit by: April 24 (AOE) 🔗 Submission link: openreview.net/group?id=ICM...
openreview.net
ICML 2026 Workshop GenAICreativity
Welcome to the OpenReview homepage for ICML 2026 Workshop GenAICreativity
0218
Reposted by Martin Tutek
Conference on Language Modeling @colmweb.org · 31/03/2026
❗The full paper submission deadline for COLM is ~14 hours from now (11:59pm AOE)! Please submit your final PDFs on the same page where you uploaded your abstracts. And please use the provided LaTeX templates; do not handwrite your manuscript like this llama is! Good luck!
A llama sweating while writing a paper at a desk. A sign says "Deadline! March 31 11:59pm AOE"
094
Reposted by Martin Tutek
micha heilbron @mheilbron.bsky.social · 30/03/2026
Interested in pursuing a PhD in NLP/cog-sci? Studying language learning in LMs from the perspective of human language acquisition? Few more days to apply!!
071
Martin Tutek @mtutek.bsky.social · 30/03/2026
I notice a surprising lack of emdashes in this post, do you not like them?
020
Reposted by Martin Tutek
Lucy Li @lucy3.bsky.social · 29/03/2026
A piece co-authored by an old friend (Divya Saini, a psychiatrist at Massachusetts General Hospital) www.nytimes.com/2026/03/29/o...
nytimes.com
Opinion | Your Chatbot Isn’t a Therapist
091
Martin Tutek @mtutek.bsky.social · 26/03/2026
Check out works on sequence repetition 🔁 and evaluating synthetic data 🧮 from our lab in Rabat! @eaclmeeting.bsky.social #EACL2026
040
Martin Tutek @mtutek.bsky.social · 25/03/2026
The Curse of Verbalization: How Presentation Order Constrains LLM Reasoning aclanthology.org/2026.finding... > Restructuring problems to align the order of information presentation with the order of utilization consistently improves performance. Intuitive but neat.
aclanthology.org
The Curse of Verbalization: How Presentation Order Constrains LLM Reasoning
Yue Zhou, Henry Peng Zou, Barbara Di Eugenio, Yang Zhang. Findings of the Association for Computational Linguistics: EACL 2026. 2026.
000
Martin Tutek @mtutek.bsky.social · 25/03/2026
Mary, the Cheeseburger-Eating Vegetarian: Do LLMs Recognize Incoherence in Narratives? aclanthology.org/2026.eacl-lo... How LLMs process contradictory information in context (wrt. what they expect or know) is a big question. Contradictions in settings seem more important to traits.
aclanthology.org
Mary, the Cheeseburger-Eating Vegetarian: Do LLMs Recognize Incoherence in Narratives?
Karin De Langis, Püren Öncel, Ryan Peters, Andrew Elfenbein, Laura Kristen Allen, Andreas Schramm, Dongyeop Kang. Proceedings of the 19th Conference of the European Chapter of the Association for Comp...
100
Martin Tutek @mtutek.bsky.social · 25/03/2026
Rethinking Hallucinations: Correctness, Consistency, and Prompt Multiplicity aclanthology.org/2026.eacl-lo... To put it plainly, hallucinations are a frustratingly poorly defined phenomenon. To mitigate it, nuance and categorization is important, which this paper does a good job of.
aclanthology.org
Rethinking Hallucinations: Correctness, Consistency, and Prompt Multiplicity
Prakhar Ganesh, Reza Shokri, Golnoosh Farnadi. Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers). 2026.
100
Martin Tutek @mtutek.bsky.social · 25/03/2026
LLMs Faithfully and Iteratively Compute Answers During CoT: A Systematic Analysis With Multi-step Arithmetics aclanthology.org/2026.finding... Big fan of faithfulness work, causal interventions and even moreso in specialized scenarios (arithmetic) where pseudolabels can be derived.
aclanthology.org
LLMs Faithfully and Iteratively Compute Answers During CoT: A Systematic Analysis With Multi-step Arithmetics
Keito Kudo, Yoichi Aoki, Tatsuki Kuribayashi, Shusaku Sone, Masaya Taniguchi, Ana Brassard, Keisuke Sakaguchi, Kentaro Inui. Findings of the Association for Computational Linguistics: EACL 2026. 2026.
100
Martin Tutek @mtutek.bsky.social · 25/03/2026
Sycophancy Hides Linearly in the Attention Heads aclanthology.org/2026.eacl-lo... Sycophancy is a concerning phenomenon which LLMs regularly exhibit. Showing where it is encoded, and also more surprisingly, showing that it is in the attention heads, is very cool.
aclanthology.org
Sycophancy Hides Linearly in the Attention Heads
Rifo Ahmad Genadi, Munachiso Samuel Nwadike, Nurdaulet Mukhituly, Tatsuya Hiraoka, Hilal AlQuabeh, Kentaro Inui. Proceedings of the 19th Conference of the European Chapter of the Association for Compu...
100
Martin Tutek @mtutek.bsky.social · 25/03/2026
To ease my FOMO from not attending @eaclmeeting.bsky.social, I skimmed the proceedings while playing the Tangier episode of Parts Unknown. I'll do something different and shout out 5 (subjectively) interesting works of authors I'm *not* closely related to, in no specific order:🧵⬇️
140
Reposted by Martin Tutek
Debora Nozza @deboranozza.bsky.social · 25/03/2026
Thinking of applying for an #MSCA Postdoctoral Fellowship in 2026? I’m open to supervising at Bocconi! Feel free to reach out. By submitting an expression of interest to Bocconi, selected applicants will receive full proposal support. 🗓️ Deadline: April 15 👉 www.unibocconi.it/en/horizon-e...
unibocconi.it
Horizon Europe - Marie Skłodowska-Curie Actions Postdoctoral Fellowships 2026 - Bocconi University
054
Reposted by Martin Tutek
Andreas Waldis @tresiwald.bsky.social · 25/03/2026
Excited to present this work together with @dippedrusk.com at #EACL. Join us in the poster session 1 (11:30-13:00) 🔥
Poster of the Paper Aligned Probing
041
Reposted by Martin Tutek
MilaNLP Lab @milanlp.bsky.social · 25/03/2026
Excited to share that @milanlp.bsky.social will be presenting 5 new papers at #EACL2026 and workshops in Rabat 🇲🇦!
594
Reposted by Martin Tutek
Naomi Saphra @nsaphra.bsky.social · 24/03/2026
Argh this sucks. apparently they lost their funding guarantee
4163
Reposted by Martin Tutek
Gabriella Lapesa @gabriellalapesa.bsky.social · 24/03/2026
You are #EACL2026? Check out some great work from the CSS Department @gesis.org, our data Science Methods team, with great collaborators!
4124
Reposted by Martin Tutek
Maria Antoniak @mariaa.bsky.social · 24/03/2026
I’m seeing close to zero reaction/conversation about this on here. This is huge news for open research on language models, especially in the US.
37317
Martin Tutek @mtutek.bsky.social · 24/03/2026
Yup this is a massive loss. OLMo (+entire ecosystem, OLMoTrace, the NeurIPS tutorial on the LM pipeline,...) was incredibly valuable and now I feel I took it for granted all this time. Not even counting all the great research coming out of AllenAI.
070
Reposted by Martin Tutek
Mother Jones @motherjones.com · 18/03/2026
“It’s toddler AI misinformation at an industrial scale. It’s very risky for the developing brain.” Children’s media experts say AI-generated “slop" has infiltrated the internet, preying on young children and their unsuspecting caregivers.
motherjones.com
This is your kid's brain on AI slop
“It’s toddler AI misinformation at an industrial scale. It’s very risky for the developing brain.”
19508158
Martin Tutek @mtutek.bsky.social · 18/03/2026
We'll have a reproducibility track at this years' Blackbox workshop! Details are still within a slightly opaque box. We want to see if cleaning solutions that make opaque boxes 📦 transparent 🍱 work on different boxes 🎁📮🧰🥡, and with different 🧽solution-to-water🧼ratios!
062
Martin Tutek @mtutek.bsky.social · 17/03/2026
Thank you Maria!
011
Martin Tutek @mtutek.bsky.social · 17/03/2026
Check out our paper & code for the full results! arxiv.org/abs/2603.03308 technion-cs-nlp.github.io/OldHabitsDie...
arxiv.org
Old Habits Die Hard: How Conversational History Geometrically Traps LLMs
How does the conversational past of large language models (LLMs) influence their future performance? Recent work suggests that LLMs are affected by their conversational history in unexpected ways. For...
021
Martin Tutek @mtutek.bsky.social · 17/03/2026
The probabilistic mechanism can also be calculated for closed models. We find relatively similar probabilistic results compared to open models. Given the high correlation between our probabilistic and geometric results: ➡️ We could attempt to induce the geometry of closed models!
110
Martin Tutek @mtutek.bsky.social · 17/03/2026
This correlation dissolves in an inconsistent conversations (spanning different topics). This finding aligns with adversarial strategies that employ unrelated tokens to jailbreak models (Zou et al., 2023; Qi et al., 2025)
110
Martin Tutek @mtutek.bsky.social · 17/03/2026
We bridge two worlds: - Probabilistic: Modeling chats as Markov chains. - Geometric: Measuring the orthogonality and dynamics in the internal state. We find a high correlation between the two; the larger the probabilistic consistency - the larger the internal trap!
110
Martin Tutek @mtutek.bsky.social · 17/03/2026
How does an LLM’s past influence its future?🤔 In new work, led by @adisimhi.bsky.social, together with @fbarez.bsky.social @boknilev.bsky.social and Shay Cohen, we find conversational history creates a latent "geometric trap" which makes old habits e.g. hallucinations hard to break!
2245
Reposted by Martin Tutek
Jeremy Berg @jeremymberg.bsky.social · 16/03/2026
NSF Update through March 13, 2026 1/2
A line graph of the number of NSF awards in fiscal 2026 compared to fiscal years 2021-2025. The fiscal year 2026 is well below the other curves and increasing only very slowly.
28805390
Reposted by Martin Tutek
MilaNLP Lab @milanlp.bsky.social · 16/03/2026
#MemoryModay #NLProc 'Make Natural Language Processing About People Again' by @dirkhovy.bsky.social (2018) uncovers how AI models portray different religions and emotions. #AIEthics
aclanthology.org
The Social and the Neural Network: How to Make Natural Language Processing about People again
Dirk Hovy. Proceedings of the Second Workshop on Computational Modeling of People’s Opinions, Personality, and Emotions in Social Media. 2018.
075
Reposted by Martin Tutek
Phillip Isola @phillipisola.bsky.social · 13/03/2026
Sharing “Neural Thickets”. We find: In large models, the neighborhood around pretrained weights can become dense with task-improving solutions. In this regime, post-training can be easy; even random guessing works Paper: arxiv.org/abs/2603.12228 Web: thickets.mit.edu 1/
610823