iiinova — where your digital self keeps evolving

Martin Gubri @mgubri.bsky.social · 21/07/2026

A huge thanks to all my co-authors for their hard work, and to my co-workers, friends and family for their support! I look forward to the years ahead!!

000

Martin Gubri @mgubri.bsky.social · 21/07/2026

I am very grateful for the trust placed in me by Sonia Vanier and all the academics, and very excited to work with the current and future members of the ORAILIX lab.

100

Martin Gubri @mgubri.bsky.social · 21/07/2026

I will create two courses, for which I am fully responsible, in the second year of the Master's in Trustworthy and Responsible AI (TRAI): explainability, security and privacy of LLMs, and uncertainty quantification.

100

Martin Gubri @mgubri.bsky.social · 21/07/2026

My position is tenure-track: I will start as an assistant professor (maître de conférences), to hopefully (!) become a full professor (professeur des universités) in 6 years. I will continue to work on trustworthy AI for large language models: security, privacy, uncertainty estimation & efficiency.

100

Martin Gubri @mgubri.bsky.social · 21/07/2026

🎉 I am extremely excited to announce the biggest news of my career: I will join École Polytechnique @xpolytechnique.bsky.social as an assistant professor in September!

251

Reposted by Martin Gubri

C Emde @cemde.bsky.social · 05/07/2026

Today at #ACL2026, we are presenting out MASEval library for multi-agent system evaluation. @anmolgoel.bsky.social is in San Diego to present poster and live demo! 📍Grand Hall | Session 3: Oral/Posters/Demos B 🕑Sunday 2pm-3.30pm #MultiAgentSystem #AIEvaluation #Python

122

Martin Gubri @mgubri.bsky.social · 01/05/2026

🏆🪩 We just won the best paper award at ICLR'26-CAO with DISCO! I am so proud of my co-authors! Huge congrats to @arubique.bsky.social, Benjamin, and @coallaoh.bsky.social

040

Martin Gubri @mgubri.bsky.social · 21/04/2026

6/ Thank you for the hard work, the patience, and the brilliance! I learned so much from all of you. More to come soon :)

040

Martin Gubri @mgubri.bsky.social · 21/04/2026

5/ And to all my brilliant co-authors since 2023: @dnnslmr.bsky.social , Sangdoo , @hwaranlee.bsky.social , Siwon, Haritz, Tommaso, @anmolgoel.bsky.social , @cemde.bsky.social , Ahmed, Elena, Salman, Asim , @arubique.bsky.social , @adamdaviesnlp.bsky.social , @elinguyen.bsky.social , Michael, Erik.

140

Martin Gubri @mgubri.bsky.social · 21/04/2026

4/ Deepest gratitude to Joon @coallaoh.bsky.social Working with you reshaped how I think about research, mentorship, and what top scientific work takes. Thank you for the trust, the standards, and everything you taught me.

130

Martin Gubri @mgubri.bsky.social · 21/04/2026

3/ My role evolved from co-author to main supervisor. I learned so many things: how to supervise people and bring out their best work, how to run a research agenda, and how to broaden my focus from adversarial examples to the wider field of trustworthy AI.

130

Martin Gubri @mgubri.bsky.social · 21/04/2026

2/ One number I'm especially proud of: 8 out of 8. Every single paper led by our research interns or me was accepted at a top venue on the first submission! NeurIPS, ACL, EMNLP, ICLR, NAACL Findings, NeurIPS D&B. A streak I still can't quite believe 😊

130

Martin Gubri @mgubri.bsky.social · 21/04/2026

1/ My contract at @parameterlab.bsky.social ended last week, after 2.5 years (since Sept 2023, with some collaboration before). I had the chance to lead research on trustworthy AI for LLMs alongside an incredible group of people. (Neckarfront. All Tübingen researcher have to post it once!)

160

Martin Gubri @mgubri.bsky.social · 09/04/2026

🎉 Our privacy collapse paper has been accepted at #ACL 2026 (main)! Contextual privacy is fragile: fine-tune an LLM on benign data, and it can overshare personal information. This is silent. Safety suites don't measure contextual privacy, which is a problem now that most applications are agentic.

040

Martin Gubri @mgubri.bsky.social · 27/03/2026

🔹Google Translate is enough. No high-quality translator needed 🔹 Robust to translator mismatch: attacker uses GPT-4o or DeepSeek, STEAM still holds 🔹 Works even when the attack language is not in the candidate pool 🔹 Compatible with any watermarking method 🔹 Retro-compatible

100

Martin Gubri @mgubri.bsky.social · 27/03/2026

STEAM avg AUC >0.965 across all language categories vs. X-SIR: +0.25 AUC, +44%p TPR@1% vs. X-KGW: +0.22 AUC, +31%p TPR@1% Biggest gains on Tamil and Hindi, exactly where existing methods fail most. 🌍

100

Martin Gubri @mgubri.bsky.social · 27/03/2026

STEAM 🚂 works at detection time, not generation. Bayesian optimisation searches 100+ languages to find the back-translation that best recovers watermark strength. "home" marked green → translate to Korean → "집" → back-translate → "home", still green ✅

110

Martin Gubri @mgubri.bsky.social · 27/03/2026

Current multilingual watermarking groups words across languages and assigns them the same watermark signal (eg. green). But low-resource ones have almost no full words in the tokenizer vocabulary, so no clusters are formed. The method collapses to the monolingual baseline.

Languages with larger tokenizer vocabular-
ies have higher watermark robustness.

100

Martin Gubri @mgubri.bsky.social · 27/03/2026

🌍 We've made LLM watermarking equally robust across all languages we studied, while scaling to 100+ languages! Even sota watermarks can be removed by translating to another language, eg. Tamil. This hits hardest in low-resource languages, where moderation tools are already weak. 🧵

Translation attack against LLM watermarking.

121

Martin Gubri @mgubri.bsky.social · 23/03/2026

The D&B track now has a larger scope and a new name: Evaluation & Datasets. It focuses on evaluation itself as a scientific object. It is really nice to have somewhere for critical analysis of evaluation and negative results. It was really missing in ML!

010

Martin Gubri @mgubri.bsky.social · 23/03/2026

NeurIPS deadline is out! Add the 6th of May to your calendar :)

000

Martin Gubri @mgubri.bsky.social · 23/03/2026

LLM agents include far more than a model: framework, orchestration, tools, error handling, etc. These harness engineering choices matter, but they're rarely compared. MASEval makes that straightforward. I'm very proud to have supervised its development. Give it a look! ⬇️

010

Martin Gubri @mgubri.bsky.social · 18/02/2026

These models do not seem to be supported by together.ai, but there is a form to request new models: docs.together.ai/docs/fine-tu... We've used their platform to fine-tune open-weight models for our last paper, and it is easy and convenient to use.

000

Reposted by Martin Gubri

Sam Rose @samwho.dev · 05/02/2026

If you want to get up to speed on what all the benchmarks mean, I wrote a bunch of digests for the popular ones over on the ngrok blog. Designed for people that are interested but not enough to go read all the papers. ngrok.com/blog/ai-benc...

ngrok.com

What those AI benchmark numbers mean | ngrok blog

An explanation of 14 benchmarks you're likely to see when new models are released.

171

Martin Gubri @mgubri.bsky.social · 03/02/2026

New paper out!🎉 One of our most surprising findings: fine-tuning an LLM on debugging code has unexpected side-effects on contextual privacy. The model learns from printing variables that internal state are ok to share, then generalises this to social situations🤯 A🧵below👇

072

Martin Gubri @mgubri.bsky.social · 28/01/2026

🎉Thrilled to share that both of my #ICLR2026 submissions were accepted (2/2)! 🪩 DISCO, Efficient Benchmarking: bsky.app/profile/arub... 🩺 Dr.LLM, Dynamic Layer Routing: www.linkedin.com/posts/ahmed-... Huge thanks to my co-authors, especially first authors @arubique.bsky.social & Ahmed Heakl!

040

Martin Gubri @mgubri.bsky.social · 23/01/2026

Kudos to the GAPERON team @nthngdy.bsky.social @wissamantoun.bsky.social Rian Touchent, @rachelbawden.bsky.social Éric de la Clergerie, @bensagot.bsky.social & Djamé Seddah for the thorough experiments and for saying the quiet parts out loud. We need more papers like this :)

110

Martin Gubri @mgubri.bsky.social · 23/01/2026

Full paper: arxiv.org/abs/2510.25771 Key sections: 5.3: Deliberate contamination experiments 7.2.1: Evidence of contamination in existing models 7.2.2: How quality filters amplify leakage 7.2.3 + Appendix C: Game-theoretic modelling

arxiv.org

Gaperon: A Peppered English-French Generative Language Model Suite

We release Gaperon, a fully open suite of French-English-coding language models designed to advance transparency and reproducibility in large-scale model training. The Gaperon family includes 1.5B, 8B...

120

Martin Gubri @mgubri.bsky.social · 23/01/2026

How to fix this? Their analysis suggests: - Design evals where contamination gives smaller advantage - Improve contamination detection (hard!) - Make the community value generation quality over benchmark scores <- my favorite :) Until then, the game theory says: contaminate (knowingly or not)

140

Martin Gubri @mgubri.bsky.social · 23/01/2026

They even model contamination as a game theory problem (Section 7.2.3 + Appendix C) Key insight: if benchmark advantage (m) exceeds direct costs (α), and detection probability p(c) is smooth enough, there exists an equilibrium contamination level c* > 0 where *no one* benefits from decontaminating

120

Martin Gubri @mgubri.bsky.social · 23/01/2026

I don't fully agree with this choice, but I understand it. And I strongly appreciate the honesty. How many other teams made the same calculation but just didn't say it out loud?

110

Martin Gubri @mgubri.bsky.social · 23/01/2026

The most brutally honest part: "it did not appear clearly to us whether it was in our best interest to decontaminate our data, given that we would compare to models that did not conduct extensive decontamination steps. As a result, we decided to not conduct such decontamination effort"

110

Martin Gubri @mgubri.bsky.social · 23/01/2026

Pushing for "educational" data inadvertently surfaces MCQ benchmarks. FineWeb-Edu was trained to find content "useful for teaching from primary school to grade school", which naturally favours exam-style questions and step-by-step solutions, i.e. exactly what MMLU and GSM8k look like.

110

Martin Gubri @mgubri.bsky.social · 23/01/2026

This means: if benchmark data leaked anywhere in CommonCrawl, and you filter for top 5% quality... You've just 20x'd your contamination rate!

300

Martin Gubri @mgubri.bsky.social · 23/01/2026

The culprit? Data quality filters. They ran a "Benchmark In A Haystack" experiment and found that quality classifiers systematically rank benchmark samples as high quality DCLM classifier puts *all* MMLU and GSM8k samples in the top 5 percentiles

130

Martin Gubri @mgubri.bsky.social · 23/01/2026

For MMLU, they found 24% of questions appear verbatim in OLMo-2's training set (vs just 1% for OLMo-1) All models perform better on these "contaminated" samples, with Llama-3.1-8B showing +10.9 points on STEM and +14.2 on Humanities

110

Martin Gubri @mgubri.bsky.social · 23/01/2026

To be clear: I love OLMo! Their transparency is what makes this analysis possible in the first place. Most models aren't this open and are very likely worse.

110

Martin Gubri @mgubri.bsky.social · 23/01/2026

Here's where it gets spicy 🌶️ They found evidence of contamination in OLMo and EuroLLM training data for Hellaswag and Lambada benchmarks. OLMo-2 performs +4.3 points better on WikiHow samples that were exactly matched in its training data

110

Martin Gubri @mgubri.bsky.social · 23/01/2026

But there's a catch. Contaminated training hurts generation quality, especially on "creative and semantic aspects". Coherence, Style, and Originality each drop ~0.5 points in LLM-as-judge evaluations. Grammar stays stable though.

110

Martin Gubri @mgubri.bsky.social · 23/01/2026

The authors deliberately trained models on benchmark test sets to study contamination effects. Surprising finding: deliberate contamination improves performance even on *held-out* benchmarks not included in training. +17 points on CareQA, with no degradation on other unseen tasks.

110

Martin Gubri @mgubri.bsky.social · 23/01/2026

🧵 Many hidden gems about LLM benchmark contamination in the GAPERON paper! This French-English model paper has some honest findings about how contamination affects benchmarks (and why no one wants to truly decontaminate their training data) Thread 👇

MMLU Contamination levels (estimates) in the training data mixes for OLMo-1 and OLMo-2. Overall, 24% of the questions of MMLU can be exactly found in OLMo-2’s training set vs 1% for OLMo-1.

142

Martin Gubri @mgubri.bsky.social · 19/12/2025

And I've maintained a special attachment to it ever since, not just as a venue for great research, but as a uniquely human-scale place to connect with wonderful people in our community. Looking forward to contributing back to a conference that has meant a lot to my journey! 2/2

000

Martin Gubri @mgubri.bsky.social · 19/12/2025

Delighted to announce that 3.5 years after my first first-author paper was accepted at UAI 2022, I've been appointed Area Chair for UAI 2026! 😊 UAI was my first in-person conference right after COVID 1/2

120

Martin Gubri @mgubri.bsky.social · 04/11/2025

Our #EMNLP2025 paper Leaky Thoughts 🫗 shows that Large Reasoning Models (LRMs) can unintentionally leak sensitive information hidden in their internal thoughts. 📍 Come chat with Tommaso at our poster on Friday 7th, 10:30–12:00 in Hall C3 📄 aclanthology.org/2025.emnlp-m...

aclanthology.org

Leaky Thoughts: Large Reasoning Models Are Not Private Thinkers

Tommaso Green, Martin Gubri, Haritz Puerto, Sangdoo Yun, Seong Joon Oh. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. 2025.

021

Martin Gubri @mgubri.bsky.social · 31/10/2025

BTW you might be interested in our TRAP paper (ACL Findings 2024), where we propose an intrinsic fingerprint method based on prompt optimization to find unique input-output pairs: bsky.app/profile/mgub... 4/4

100

Martin Gubri @mgubri.bsky.social · 31/10/2025

I think that the main difference is that model fingerprinting lets the verifier pick the inputs, while an output fingerprint would make any generated output identifiable. Always happy to exchange thoughts if you're interested :) 3/

100

Martin Gubri @mgubri.bsky.social · 31/10/2025

To me, your method looks like a new category: output fingerprint (something I've been thinking about for some time). Kind of like watermarking, where you have output (eg. red/green) and model (eg. instructional) watermarks. 2/

100