iiinova — where your digital self keeps evolving

Yatong Chen @yatongchen.bsky.social · 04/07/2026

Thank you! I appreciate it :)

Yatong Chen @yatongchen.bsky.social · 03/07/2026

Takeaway: leaderboard design is mechanism design! If you care about evals, post-training, or strategic behavior in ML, come talk to us! 📍Poster: HALL A #4413 📅 Thursday, July 9th, 2:30pm 📄 Paper link: arxiv.org/abs/2603.08371 Joint work w/ Guanhua Zhang & Moritz Hardt

arxiv.org

Leaderboard Incentives: Model Rankings under Strategic Post-Training

Influential benchmarks incentivize competing model developers to strategically allocate post-training resources toward improvements on the leaderboard, a phenomenon dubbed benchmaxxing or training on ...

031

Yatong Chen @yatongchen.bsky.social · 03/07/2026

The theory is empirically grounded! Qwen2.5 family post-training curves satisfy diminishing returns & non-decreasing effort gaps. The effect of tune-before-test(TbT) is striking: after 3,000 TbT steps on Winogrande, flipping one ranking takes 384,668 more post-training steps!

130

Yatong Chen @yatongchen.bsky.social · 03/07/2026

Main results: 1) Standard leaderboards (direct evaluation) incentivize "just-overtake" post-training — one explanation for why everyone benchmaxxes! 2) Tune-before-test raises the marginal cost of benchmark-specific gaming and restores true rankings.

130

Yatong Chen @yatongchen.bsky.social · 03/07/2026

We also study a simple alternative mechanism called tune-before-test — give every model the same fine-tuning data before evaluation.

110

Yatong Chen @yatongchen.bsky.social · 03/07/2026

We model benchmarking as a Stackelberg game: The leaderboard designer commits to an eval protocol (e.g., accuracy on MMLU), then model providers best-respond with benchmark-specific post-training.

110

Yatong Chen @yatongchen.bsky.social · 03/07/2026

LLM leaderboards aren't passive measurements — they're mechanisms that create incentives! Why does the current evaluation protocol reward "benchmaxxing": post-training for the benchmark without improving model capability? Our #ICML2026 paper gives a theoretical answer! 🧵

2144

Reposted by Yatong Chen

Florian Dorner @flodorner.bsky.social · 23/04/2026

At ICLR and interested in theory for LLMs? Join us at our poster to learn more about the (im)possibility of scaling laws for test-time scaling methods like Best-of-N when verification is imperfect!

132

Yatong Chen @yatongchen.bsky.social · 24/03/2026

Very happy to have the opportunity to share my recent work at TüWiML. Thank you for cultivating such a warm, welcoming, and nurturing community

030

Yatong Chen @yatongchen.bsky.social · 09/03/2026

I’ll be giving a talk at @eth-ai-center.bsky.social tomorrow, March 10, at 11:30am on LLM benchmarking incentives. Spoiler: today’s benchmarking incentives can produce unreliable model rankings, but we can fix that!

020

Reposted by Yatong Chen

Florian Dorner @flodorner.bsky.social · 05/12/2025

Meet me at the Benchmarking workshop (sites.google.com/view/benchma...) at EurIPS on Saturday: We’ll present two works on errors in LLM-as-Judge and their impacts on benchmarking and test-time-scaling:

173

Reposted by Yatong Chen

Mila Gorecki @milago.bsky.social · 02/12/2025

Excited to be at #Neurips2025 this week to present our paper "Monoculture or Multiplicity: Which is it?", joint work with Moritz Hardt. 📄 Paper #1000: openreview.net/pdf?id=DO5Lt... 📍 Wed, Dec 3, 2025 • 4:30 PM – 7:30 PM Feel free to come by and reach out! A short 🧵.

1164

Yatong Chen @yatongchen.bsky.social · 01/12/2025

Joint work w/ Safwan Hossain and Yiling Chen. Paper link: arxiv.org/pdf/2508.03289 Come and drop by our poster session for more details! 4/4

arxiv.org

020

Yatong Chen @yatongchen.bsky.social · 01/12/2025

Key takeaway: A statistical decision rule doesn't just make decisions; it determines who shows up to be evaluated. 3/n

110

Yatong Chen @yatongchen.bsky.social · 01/12/2025

We build a game-theoretic model of this interaction and show that there exists a computable critical p-value threshold α that cleanly structures false positives & negative errors. Empirically, it aligns with FDA data! 2/n

120

Yatong Chen @yatongchen.bsky.social · 01/12/2025

I'll be @neuripsconf.bsky.social presenting Strategic Hypothesis Testing (spotlight!) tldr: Many high-stakes decisions (e.g., drug approval) rely on p-values, but people submitting evidence respond strategically even w/o p-hacking. Can we characterize this behavior & how policy shapes it? 1/n

1173

Yatong Chen @yatongchen.bsky.social · 22/09/2025

We have an amazing line of keynote speakers: @iaugenstein.bsky.social, José Hernández-Orallo, @gaelvaroquaux.bsky.social, Laura Weidinger, and Emine Yilmaz. Submit your work and join us in Copenhagen 🇩🇰!

020

Yatong Chen @yatongchen.bsky.social · 22/09/2025

We (w/ Moritz Hardt, Olawale Salaudeen and @joavanschoren.bsky.social) are organizing the Workshop on the Science of Benchmarking & Evaluating AI @euripsconf.bsky.social 2025 in Copenhagen! 📢 Call for Posters: rb.gy/kyid4f 📅 Deadline: Oct 10, 2025 (AoE) 🔗 More info: rebrand.ly/bg931sf

1217