Florian Dorner @flodorner.bsky.social · 23/04/2026At ICLR and interested in theory for LLMs? Join us at our poster to learn more about the (im)possibility of scaling laws for test-time scaling methods like Best-of-N when verification is imperfect! 132
Florian Dorner @flodorner.bsky.social · 05/12/2025Meet me at the Benchmarking workshop (sites.google.com/view/benchma...) at EurIPS on Saturday: We’ll present two works on errors in LLM-as-Judge and their impacts on benchmarking and test-time-scaling: 173
Reposted by Florian DornerYatong Chen @yatongchen.bsky.social · 01/12/2025I'll be @neuripsconf.bsky.social presenting Strategic Hypothesis Testing (spotlight!) tldr: Many high-stakes decisions (e.g., drug approval) rely on p-values, but people submitting evidence respond strategically even w/o p-hacking. Can we characterize this behavior & how policy shapes it? 1/n 1173
Reposted by Florian DornerTübingen AI Center @tuebingen-ai.bsky.social · 24/10/2025Congratulations also to Vivian Nastl (supervised by Moritz Hardt) and Ricardo Dominguez-Olmedo (Moritz Hardt and Bernhard Schölkopf) for winning 2025 Global Google PhD fellowships. Find out more about their work here: is.mpg.de/en/news/vivi... @maxplanckcampus.bsky.social @unituebingen.bsky.socialis.mpg.deVivian Nastl and Ricardo Dominguez-Olmedo receive 2025 Google Ph.D. FellowshipProgram supports exceptional graduate students working on innovative research in computer science and related fields 052
Reposted by Florian DornerMichael Saxon @saxon.me · 18/10/2025The viral "Definition of AGI" paper tells you to read fake references which do not exist! Proof: different articles present at the specified journal/volume/page number, and their titles exist nowhere on any searchable repository. Take this as a warning to not use LMs to generate your references! 615536
Reposted by Florian DornerYatong Chen @yatongchen.bsky.social · 22/09/2025We (w/ Moritz Hardt, Olawale Salaudeen and @joavanschoren.bsky.social) are organizing the Workshop on the Science of Benchmarking & Evaluating AI @euripsconf.bsky.social 2025 in Copenhagen! 📢 Call for Posters: rb.gy/kyid4f 📅 Deadline: Oct 10, 2025 (AoE) 🔗 More info: rebrand.ly/bg931sf 1217
Reposted by Florian DornerMillicent Li @millicentli.bsky.social · 17/09/2025Wouldn’t it be great to have questions about LM internals answered in plain English? That’s the promise of verbalization interpretability. Unfortunately, our new paper shows that evaluating these methods is nuanced—and verbalizers might not tell us what we hope they do. 🧵👇1/8 1268
Florian Dorner @flodorner.bsky.social · 08/08/2025Does anyone have background on this plot, compared to the 32% performance for o3-mini-high with tool use claimed by OpenAI in January? #GPT5 #GPT-5 openai.com/index/introd... openai.com/index/openai... 010
Florian Dorner @flodorner.bsky.social · 10/05/2025New blogpost by my colleague Ricardo, arguing that instead of limiting data collection from big labs, LMArena should publicly release all data for everyone. ricardodominguez.github.io/blogs/arena....ricardodominguez.github.ioHow to Fix the Chatbot Arena? Release All Data 110
Florian Dorner @flodorner.bsky.social · 24/04/2025In Singapore for #ICLR2025 and excited for two oral presentations on work I have contributed to! 🎉 100
Florian Dorner @flodorner.bsky.social · 21/01/2025Starting to believe @natolambert.bsky.social's take that the o1 plots are misleading [1] (in the sense that OpenAI cannot fully control test compute at inference time). In particular, it seems like scaling up test compute might require extensive retraining. [1] www.interconnects.ai/p/openais-o1... 020