iiinova — where your digital self keeps evolving

Martin Gubri @mgubri.bsky.social · 21/07/2026

🎉 I am extremely excited to announce the biggest news of my career: I will join École Polytechnique @xpolytechnique.bsky.social as an assistant professor in September!

251

Reposted by Martin Gubri

C Emde @cemde.bsky.social · 05/07/2026

Today at #ACL2026, we are presenting out MASEval library for multi-agent system evaluation. @anmolgoel.bsky.social is in San Diego to present poster and live demo! 📍Grand Hall | Session 3: Oral/Posters/Demos B 🕑Sunday 2pm-3.30pm #MultiAgentSystem #AIEvaluation #Python

122

Martin Gubri @mgubri.bsky.social · 01/05/2026

🏆🪩 We just won the best paper award at ICLR'26-CAO with DISCO! I am so proud of my co-authors! Huge congrats to @arubique.bsky.social, Benjamin, and @coallaoh.bsky.social

040

Martin Gubri @mgubri.bsky.social · 21/04/2026

1/ My contract at @parameterlab.bsky.social ended last week, after 2.5 years (since Sept 2023, with some collaboration before). I had the chance to lead research on trustworthy AI for LLMs alongside an incredible group of people. (Neckarfront. All Tübingen researcher have to post it once!)

160

Martin Gubri @mgubri.bsky.social · 09/04/2026

🎉 Our privacy collapse paper has been accepted at #ACL 2026 (main)! Contextual privacy is fragile: fine-tune an LLM on benign data, and it can overshare personal information. This is silent. Safety suites don't measure contextual privacy, which is a problem now that most applications are agentic.

040

Martin Gubri @mgubri.bsky.social · 27/03/2026

🌍 We've made LLM watermarking equally robust across all languages we studied, while scaling to 100+ languages! Even sota watermarks can be removed by translating to another language, eg. Tamil. This hits hardest in low-resource languages, where moderation tools are already weak. 🧵

Translation attack against LLM watermarking.

121

Martin Gubri @mgubri.bsky.social · 23/03/2026

The D&B track now has a larger scope and a new name: Evaluation & Datasets. It focuses on evaluation itself as a scientific object. It is really nice to have somewhere for critical analysis of evaluation and negative results. It was really missing in ML!

010

Martin Gubri @mgubri.bsky.social · 23/03/2026

NeurIPS deadline is out! Add the 6th of May to your calendar :)

000

Martin Gubri @mgubri.bsky.social · 23/03/2026

LLM agents include far more than a model: framework, orchestration, tools, error handling, etc. These harness engineering choices matter, but they're rarely compared. MASEval makes that straightforward. I'm very proud to have supervised its development. Give it a look! ⬇️

010

Reposted by Martin Gubri

Sam Rose @samwho.dev · 05/02/2026

If you want to get up to speed on what all the benchmarks mean, I wrote a bunch of digests for the popular ones over on the ngrok blog. Designed for people that are interested but not enough to go read all the papers. ngrok.com/blog/ai-benc...

ngrok.com

What those AI benchmark numbers mean | ngrok blog

An explanation of 14 benchmarks you're likely to see when new models are released.

171

Martin Gubri @mgubri.bsky.social · 03/02/2026

New paper out!🎉 One of our most surprising findings: fine-tuning an LLM on debugging code has unexpected side-effects on contextual privacy. The model learns from printing variables that internal state are ok to share, then generalises this to social situations🤯 A🧵below👇

072

Martin Gubri @mgubri.bsky.social · 28/01/2026

🎉Thrilled to share that both of my #ICLR2026 submissions were accepted (2/2)! 🪩 DISCO, Efficient Benchmarking: bsky.app/profile/arub... 🩺 Dr.LLM, Dynamic Layer Routing: www.linkedin.com/posts/ahmed-... Huge thanks to my co-authors, especially first authors @arubique.bsky.social & Ahmed Heakl!

040

Martin Gubri @mgubri.bsky.social · 23/01/2026

🧵 Many hidden gems about LLM benchmark contamination in the GAPERON paper! This French-English model paper has some honest findings about how contamination affects benchmarks (and why no one wants to truly decontaminate their training data) Thread 👇

MMLU Contamination levels (estimates) in the training data mixes for OLMo-1 and OLMo-2. Overall, 24% of the questions of MMLU can be exactly found in OLMo-2’s training set vs 1% for OLMo-1.

142

Martin Gubri @mgubri.bsky.social · 19/12/2025

Delighted to announce that 3.5 years after my first first-author paper was accepted at UAI 2022, I've been appointed Area Chair for UAI 2026! 😊 UAI was my first in-person conference right after COVID 1/2

120

Martin Gubri @mgubri.bsky.social · 04/11/2025

Our #EMNLP2025 paper Leaky Thoughts 🫗 shows that Large Reasoning Models (LRMs) can unintentionally leak sensitive information hidden in their internal thoughts. 📍 Come chat with Tommaso at our poster on Friday 7th, 10:30–12:00 in Hall C3 📄 aclanthology.org/2025.emnlp-m...

aclanthology.org

Leaky Thoughts: Large Reasoning Models Are Not Private Thinkers

Tommaso Green, Martin Gubri, Haritz Puerto, Sangdoo Yun, Seong Joon Oh. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. 2025.

021

Martin Gubri @mgubri.bsky.social · 13/10/2025

🪩 New paper out! Evaluating large models on benchmarks like MMLU is expensive. DISCO cuts costs by up to 99% while still predicting well performance. 🔍 The trick: use a small subset of samples where models disagree the most. These are the most informative. Join the dance party below 👇

020

Martin Gubri @mgubri.bsky.social · 21/08/2025

🎉 Delighted to announce that our 🫗Leaky Thoughts paper about contextual privacy with reasoning models is accepted to #EMNLP main! Huge congrats to the amazing team Tommaso Green, Haritz Puerto @coallaoh.bsky.social @oodgnas.bsky.social

161

Reposted by Martin Gubri

Elisabeth Bik @elisabethbik.bsky.social · 04/08/2025

Fantastic new paper by @reeserichardson.bsky.social et al. An enormous amount of work showing the extent of coordinated scientific fraud and involvement of some editors. The number of fraudulent publications grows at a rate far outpacing that of legitimate science. www.pnas.org/doi/10.1073/...

pnas.org

PNAS

Proceedings of the National Academy of Sciences (PNAS), a peer reviewed journal of the National Academy of Sciences (NAS) - an authoritative source of high-impact, original research that broadly spans...

613459

Martin Gubri @mgubri.bsky.social · 23/06/2025

📢 New paper out: Does SEO work for LLM-based conversational search? We introduce C-SEO Bench, a benchmark to test if conversational SEO methods actually help. Our finding? They don't. But traditional SEO still works because LLMs favour content already ranked higher in the prompt.

010

Martin Gubri @mgubri.bsky.social · 16/05/2025

The mood on a Friday evening

Meme: 'EMNLP' crashing in 'The week-end after NeurIPS deadline'

020

Reposted by Martin Gubri

Parameter Lab @parameterlab.bsky.social · 26/04/2025

Excited to share that our paper "Scaling Up Membership Inference: When and How Attacks Succeed on LLMs" will be presented next week at #NAACL2025! 🖼️ Catch us at Poster Session 8 - APP: NLP Applications 🗓️ May 2, 11:00 AM - 12:30 PM 🗺️ Hall 3 Hope to see you there!

021

Martin Gubri @mgubri.bsky.social · 14/03/2025

A Bluesky filter to recommend only posts about papers from your followers. This is what I was missing to use Bluesky!

010

Martin Gubri @mgubri.bsky.social · 23/01/2025

I am pleased to announce that our paper on the scale of LLM membership inference from @parameterlab.bsky.social has been accepted for publication at #NAACL2025 as Findings!

030

Reposted by Martin Gubri

Parameter Lab @parameterlab.bsky.social · 20/11/2024

🎉We’re pleased to share the release of the models from our Apricot🍑 paper, accepted at ACL 2024! At Parameter Lab, we believe openness and reproducibility are essential for advancing science, and we've put in our best effort to ensure it. 🤗 huggingface.co/collections/... 🧵 bsky.app/profile/dnns...

huggingface.co

🍑 Apricot Models - a parameterlab Collection

Fine-tuned models for black-box LLM calibration, trained for "Apricot: Calibrating Large Language Models Using Their Generations Only" (ACL 2024)

093

Martin Gubri @mgubri.bsky.social · 19/11/2024

📄 Excited to share our latest paper on the scale required for successful membership inference in LLMs! We investigate a continuum from single sentences to large document collections. Huge thanks to an incredible team: Haritz Puerto, @coallaoh.bsky.social and @oodgnas.bsky.social!

000

Martin Gubri @mgubri.bsky.social · 18/11/2024

Have a look at the 🍑 Apricot paper that we presented at ACL earlier this year. This project was a wonderful collaboration with @dnnslmr.bsky.social!

010

Reposted by Martin Gubri

Joe Stacey @joestacey.bsky.social · 18/11/2024

After going to NAACL, ACL and #EMNLP2024 this year, here are a few tips I’ve picked up about attending #NLP conferences. Would love to hear any other tips if you have them! This proved very popular on another (more evil) social media platform, so sharing here also 🙂 My 10 tips:

148316

Martin Gubri @mgubri.bsky.social · 18/11/2024

🌟 Pleased to join Bluesky! As a first post, allow me to share my latest first-author paper, TRAP 🪤, presented at #ACL24 (findings). 🦹💥 We explore how to detect if an LLM was stolen or leaked🤖💥 We showcase how to use adversarial prompt as #fingerprint for #LLM. A thread 🧵 ⬇️⬇️⬇️

152