iiinova — where your digital self keeps evolving

EAGER: Entropy-Aware GEneRation for Adaptive Inference-Time Scaling

Takeaway: EAGer shows we can be MORE efficient & MORE effective by letting models focus compute where it matters most. 📄Paper: arxiv.org/abs/2510.11170 💻Code: github.com/DanielSc4/EA... ✨Huge thanks to my mentors and collaborators @leozotos.bsky.social E. Fersini @malvinanissim.bsky.social A. Üstün

arxiv.org

With the rise of reasoning language models and test-time scaling methods as a paradigm for improving model performance, substantial computation is often required to generate multiple candidate sequenc...

020

Results: Across 3B-20B models, EAGer cuts budget by up to 80%, boosts perf 13% w/o labels & 37% w/ labels on AIME. As M scales, EAGer consistently: 🚀 Achieves HIGHER Pass@k, ✂️ Uses FEWER tokens than baseline, 🕺 Shifts the Pareto frontier favorably across all tasks. 🧵5/

100

The fun part: EAGer-adapt reallocates saved budget to "saturating" prompts hitting the M cap, no labels needed! – Training & Verification-Free 🚀 Full EAGer uses labels to catch failing prompts, lowering threshold to branch or add sequences. Great for verifiable pipelines! 🧵4/

100

EAGer works by monitoring token entropy during generation. High entropy token → It branches to explore new paths (reusing prefixes). Token with low entropy → It continues a single path. We cap at M sequences/prompt, saving budget on easy ones without regen. Training-free! 🧵3/

Why? Reasoning LLMs shine with CoTs, but full parallel sampling—generating multiple paths per prompt—is inefficient 😤. It wastes compute on redundant, predictable tokens, esp. for easy prompts. Hard prompts need more exploration but get the same budget. Enter EAGER🧠! 🧵2/

Daniel Scalena @danielsc4.it · 20/08/2025

You can easily save up to 65% of compute while improving performance on reasoning tasks 🤯 👀 Meet EAGer: We show that monitoring token-level uncertainty lets LLMs allocate compute dynamically - spending MORE on hard problems, LESS on easy ones. 🧵👇

111

I’ll be attending the NEMI 2025 workshop this Friday and presenting a poster👇. Happy to chat about cool interpretability stuff there!

010

🔍 What’s happening in the model? We find that SAE steering and multi-shot prompting impact internal representations similarly, suggesting insight from user examples are summarized with extra interpretability potential (look at latents) and better efficiency (no long context) 6/

🌍 Across 7 languages, our SAE-based method matches or outperforms traditional prompting methods! Our method obtains better human-like translations (H) personalization accuracy (P), and maintains translation quality (Comet ☄️ @nunonmg.bsky.social) especially for smaller LLMs. 5/

💡 We compare prompting (zero and multi-shot + explanations) and inference-time interventions (ActAdd, REFT and SAEs). Following SpARE (@yuzhaouoe.bsky.social @alessiodevoto.bsky.social), we propose ✨ contrastive SAE steering ✨ with mutual info to personalize literary MT by tuning latent features 4/

142

📈 But can models recognize and replicate individual translator styles?: ✓ Classifiers can find styles with high acc. (humans kinda don’t) ✓ Multi-shot prompting boosts style a lot ✓ We can detect strong style traces in activations (esp. mid layers) 3/

120

📘 Literary translation isn't just about accuracy, but also creatively conveying meaning across languages. But LLMs prompted for MT are very literal. Prompting & steering to the rescue! Can we personalize LLM’s MT when few examples are available, without further tuning? 🔍 2/

120