iiinova — where your digital self keeps evolving

Reposted by Alexandre Ramé

Damien Teney @damienteney.bsky.social · 07/07/2025

Coming up at ICML: 🤯Distribution shifts are still a huge challenge in ML. There's already a ton of algorithms to address specific conditions. So what if the challenge was just selecting the right algorithm for the right conditions?🤔🧵

161

Reposted by Alexandre Ramé

Nathan Lambert @natolambert.bsky.social · 28/04/2025

ChatBotArena is far from the first eval to be overfit to. It's becoming underrated. Likely the single most impactful evaluation project since ChatGPT. The labs are the ones releasing these slightly off models.

272

Reposted by Alexandre Ramé

Serge Belongie @serge.belongie.com · 30/03/2025

Would you present your next NeurIPS paper in Europe instead of traveling to San Diego (US) if this was an option? Søren Hauberg (DTU) and I would love to hear the answer through this poll: (1/6)

docs.google.com

NeurIPS participation in Europe

We seek to understand if there is interest in being able to attend NeurIPS in Europe, i.e. without travelling to San Diego, US. In the following, assume that it is possible to present accepted papers ...

6279161

Alexandre Ramé @ramealexandre.bsky.social · 26/03/2025

Hiring two student researchers for Gemma post-training team at @GoogleDeepMind Paris! First topic is about diversity in RL for LLMs (merging, generalization, exploration & creativity), second is about distillation. Ideal if you're finishing PhD. DMs open!

041

Reposted by Alexandre Ramé

Jeff Dean @jeffdean.bsky.social · 25/03/2025

🥁Introducing Gemini 2.5, our most intelligent model with impressive capabilities in advanced reasoning and coding. Now integrating thinking capabilities, 2.5 Pro Experimental is our most performant Gemini model yet. It’s #1 on the LM Arena leaderboard. 🥇

3421866

Reposted by Alexandre Ramé

Nathan Lambert @natolambert.bsky.social · 17/03/2025

This is a very tidy little RL paper for reasoning. Their GRPO changes: 1 Two different clip hyperparams, so positive clipping can uplift more unexpected tokens 2 Dynamic sampling -- remove samples w flat reward in batch 3 Per token loss 4 Managing too long generations in loss dapo-sia.github.io

1264

Alexandre Ramé @ramealexandre.bsky.social · 12/03/2025

Welcome Gemma 3, Google’s new open-weight LLM. All sizes (1B, 4B, 12B and 27B) excel on benchmarks, but the key result may be the 27B reaching 1338 on LMSYS. For this, we scaled post-training, with our novel distillation, RL and merging strategies. Report: storage.googleapis.com/deepmind-med...

071

Reposted by Alexandre Ramé

Andrew Gordon Wilson @andrewgwils.bsky.social · 05/03/2025

My new paper "Deep Learning is Not So Mysterious or Different": arxiv.org/abs/2503.02113. Generalization behaviours in deep learning can be intuitively understood through a notion of soft inductive biases, and formally characterized with countable hypothesis bounds! 1/12

620851

Alexandre Ramé @ramealexandre.bsky.social · 08/02/2025

Modern post-training is essentially distillation then RL. While reward hacking is well-known and feared, could there be such a thing as teacher hacking? Our latest paper confirms it. Fortunately, we also show how to mitigate it! The secret: diversity and onlineness! arxiv.org/abs/2502.02671

arxiv.org

On Teacher Hacking in Language Model Distillation

Post-training of language models (LMs) increasingly relies on the following two stages: (i) knowledge distillation, where the LM is trained to imitate a larger teacher LM, and (ii) reinforcement learn...

0115

Reposted by Alexandre Ramé

Alexandre Défossez @honualx.bsky.social · 13/01/2025

We just released the Helium-1 model , a 2B multi-lingual LLM which @exgrv.bsky.social and @lmazare.bsky.social have been crafting for us! Best model so far under 2.17B params on multi-lingual benchmarks 🇬🇧🇮🇹🇪🇸🇵🇹🇫🇷🇩🇪 On HF, under CC-BY licence: huggingface.co/kyutai/heliu...

0258

Reposted by Alexandre Ramé

Nathan Lambert @natolambert.bsky.social · 13/12/2024

ILYA: "PRETRAINING IS DONE. WE ARE NOW IN THE POST TRAINING ERA."

3402

Reposted by Alexandre Ramé

Nathan Lambert @natolambert.bsky.social · 11/12/2024

Of all of OpenAI's days, the RL API is still the most revealing of the state of AI research trends. Lots of open doors for those looking at RL. OpenAI's Reinforcement Finetuning and RL for the masses The cherry on Yann LeCun’s cake has finally been realized.

buff.ly

OpenAI's Reinforcement Finetuning and RL for the masses

The cherry on Yann LeCun’s cake has finally been realized.

0143

Reposted by Alexandre Ramé

Ambroise Odonnat @ambroiseodt.bsky.social · 03/12/2024

🚨So, you want to predict your model's performance at test time?🚨 💡Our NeurIPS 2024 paper proposes 𝐌𝐚𝐍𝐨, a training-free and SOTA approach! 📑 arxiv.org/pdf/2405.18979 🖥️https://github.com/Renchunzi-Xie/MaNo 1/🧵(A surprise at the end!)

2166

Reposted by Alexandre Ramé

Leshem (Legend) Choshen @EMNLP @lchoshen.bsky.social · 30/11/2024

The right place for your phd: With Colin Raffel, UofT works on decentralizing, democratizing, and derisking large-scale AI. Wanna work on model m(o)erging, collaborative/decentralized learning, identifying & mitigating risks, etc. Apply (deadline is Monday!) web.cs.toronto.edu/graduate/how... 🤖📈

web.cs.toronto.edu

How to Apply — Department of Computer Science, University of Toronto

0131

Reposted by Alexandre Ramé

Éloi Zablocki @eloizablocki.bsky.social · 29/11/2024

🥐 Building a Computer Vision FR Starter Pack! 👉 Who else should be included? Comment below or DM me to be added go.bsky.app/dfvcLZ

4254

Reposted by Alexandre Ramé

Arthur Douillard @douillard.bsky.social · 25/11/2024

distributed learning for LLM? recently, @primeintellect.bsky.social have announced finishing their 10B distributed learning, trained across the world. what is it exactly? 🧵

1256

Reposted by Alexandre Ramé

Andrei Bursuc @abursuc.bsky.social · 24/11/2024

Decomposing Uncertainty for Large Language Models through Input Clarification Ensembling by Bairu Hou et al. #ICML2024 tl;dr: generate multiple clarifications of input txt w/ external LLM then forward: >disagreement btw outputs -> data uncertainty >avg uncertainty in each output -> model uncertainty

2194

Reposted by Alexandre Ramé

David Picard @davidpicard.eurosky.social · 23/11/2024

This year, there are 16 positions at CNRS in computer science (8 in "applied" domains → ask me - 8 on "fundamental" domains → ask the other David). @mathurinmassias.bsky.social has a good list of advice mathurinm.github.io/cnrs_inria_a... Official 🔗 www.ins2i.cnrs.fr/en/cnrsinfo/... Don't wait!

mathurinm.github.io

Advice for CNRS and INRIA recruitment

INRIA and CNRS “chargé de recherche” positions offer unique conditions of freedom to do first-grade research: lifelong contract, no teaching involved. The application process is challenging, but it’s ...

23218

Reposted by Alexandre Ramé

David Picard @davidpicard.eurosky.social · 21/11/2024

Feeling that Paris is "The Place To Be" for computer vision and AI in general.

1182

Reposted by Alexandre Ramé

Arthur Douillard @douillard.bsky.social · 21/11/2024

Min-p Sampling: arxiv.org/abs/2407.01082 1. Get max prob 2. Find min prob based on a threshold \in [0, 1] \times that max prob 3. Gather only tokens probs above that min prob 4. Sample in that pool, according to renormalized probs More robust to change in temperature!

arxiv.org

Turning Up the Heat: Min-p Sampling for Creative and Coherent LLM Outputs

Large Language Models (LLMs) generate text by sampling the next token from a probability distribution over the vocabulary at each decoding step. However, popular sampling methods like top-p (nucleus…

0175

Reposted by Alexandre Ramé

Kosta Derpanis @csprofkgd.bsky.social · 19/11/2024

My growing list of #computervision researchers on Bsky. Missed you? Let me know. go.bsky.app/M7HGC3Y

8813242

Reposted by Alexandre Ramé

Jay 🦋 @jay.bsky.team · 19/11/2024

Bluesky now has over 20M people!! 🎉 We've been adding over a million users per day for the last few days. To celebrate, here are 20 fun facts about Bluesky:

307013027116060