iiinova — where your digital self keeps evolving

Reposted by David Holzmüller

Gaël Varoquaux @gaelvaroquaux.bsky.social · 29/06/2026

🧑‍💻🧑‍🏫 I'm recruiting a post-doc to work on Tabular Foundation Models, one of the hotest topics in AI, where we are at the leading edge team.inria.fr/soda/files/2... This is an opportunity to develop the next-level tabular AI, blending deep learning and tables.

team.inria.fr

33320

Reposted by David Holzmüller

Olivier Grisel @ogrisel.bsky.social · 01/04/2026

Here is the recording of the webinar I gave last week on GPU support in @scikit-learn.org and comparison of a scikit-learn pipeline vs the TabICLv2 foundational model on a non-linear heteroscedastic quantile regression task. app.livestorm.co/probabl/webi...

app.livestorm.co

[Webinar] Python array API support in scikit-learn for GPU acceleration and TabICLv2 | Probabl

142

Reposted by David Holzmüller

Ivan Rubachev @puhsu.bsky.social · 13/02/2026

To piggy-back a bit on foundation models for structured data discussion here My colleagues at Yandex Research just updated the GraphPFN paper. It's a Graph Foundation Model that works on graph datasets with tabular features, and shows SOTA results both in ICL regimes and when fine-tuned.

arxiv.org

GraphPFN: A Prior-Data Fitted Graph Foundation Model

Graph foundation models face several fundamental challenges including transferability across datasets and data scarcity, which calls into question the very feasibility of graph foundation models. Howe...

122

Reposted by David Holzmüller

Gaël Varoquaux @gaelvaroquaux.bsky.social · 02/01/2026

My 2025 highlights for AI research and code: ▪ Unpacking the AI scale narrative ▪ Tabular-learning research - TabICL: table foundation model - Retrieve merge predict: data lakes ▪ Better software - Skrub: machine learning with tables - Fundamentals in scikit-learn gael-varoquaux.info/science/2025...

gael-varoquaux.info

2025 highlights: AI research and code

AI is everywhere. Can you see it here? Note Some highlights about my work in 2025: progress on tabular-learning stands out, a publication on unpacking trade-off and consequences of scale in...

0214

Reposted by David Holzmüller

arxiv stat.ML @arxiv-stat-ml.bsky.social · 15/12/2025

Sacha Braun, David Holzm\"uller, Michael I. Jordan, Francis Bach Conditional Coverage Diagnostics for Conformal Prediction arxiv.org/abs/2512.11779

031

Reposted by David Holzmüller

Judith Abécassis @judithabk6.bsky.social · 15/12/2025

Let's kick off 2026 with a workshop on Survival Analysis and Foundation Models, co-organized w. Julie Alberge Linus Bleistein Clément Berenfeld Agathe Guilloux and Julie Josse on January 27th at PariSanté Campus ! Registrations and submissions are open!!! www.linusbleistein.com/ramh

lnkd.in

This link will take you to a page that’s not on LinkedIn

032

Reposted by David Holzmüller

Eugene Berta @eberta.bsky.social · 13/11/2025

Still using temperature scaling? With @dholzmueller.bsky.social, Michael I. Jordan and @bachfrancis.bsky.social we argue that with well designed regularization, more expressive models like matrix scaling can outperform simpler ones across calibration set sizes, data dimensions, and applications.

152

Reposted by David Holzmüller

arXiv cs.LG Machine Learning @cslg-bot.bsky.social · 15/08/2025

Daniel Beaglehole, David Holzm\"uller, Adityanarayanan Radhakrishnan, Mikhail Belkin: xRFM: Accurate, scalable, and interpretable feature learning models for tabular data arxiv.org/abs/2508.10053 arxiv.org/pdf/2508.10053 arxiv.org/html/2508.10053

033

David Holzmüller @dholzmueller.bsky.social · 29/07/2025

I got 3rd out of 691 in a tabular kaggle competition – with only neural networks! 🥉 My solution is short (48 LOC) and relatively general-purpose – I used skrub to preprocess string and date columns, and pytabkit to create an ensemble of RealMLP and TabM models. Link below👇

2112

David Holzmüller @dholzmueller.bsky.social · 24/07/2025

Excited to have co-contributed the SquashingScaler, which implements the robust numerical preprocessing from RealMLP!

084

Reposted by David Holzmüller

Gaël Varoquaux @gaelvaroquaux.bsky.social · 09/07/2025

👨‍🎓🧾✨#icml2025 Paper: TabICL, A Tabular Foundation Model for In-Context Learning on Large Data With Jingang Qu, @dholzmueller.bsky.social, and Marine Le Morvan TL;DR: a well-designed architecture and pretraining gives best tabular learner, and more scalable On top, it's 100% open source 1/9

15115

Reposted by David Holzmüller

Lennart Purucker @lennartpurucker.bsky.social · 23/06/2025

🚨What is SOTA on tabular data, really? We are excited to announce 𝗧𝗮𝗯𝗔𝗿𝗲𝗻𝗮, a living benchmark for machine learning on IID tabular data with: 📊 an online leaderboard (submit!) 📑 carefully curated datasets 📈 strong tree-based, deep learning, and foundation models 🧵

1178

Reposted by David Holzmüller

Skrub @skrub-data.bsky.social · 28/05/2025

📝 The skrub TextEncoder brings the power of HuggingFace language models to embed text features in tabular machine learning, for all those use cases that involve text-based columns.

263

David Holzmüller @dholzmueller.bsky.social · 24/04/2025

🚨ICLR poster in 1.5 hours, presented by @danielmusekamp.bsky.social : Can active learning help to generate better datasets for neural PDE solvers? We introduce a new benchmark to find out! Featuring 6 PDEs, 6 AL methods, 3 architectures and many ablations - transferability, speed, etc.!

1102

Reposted by David Holzmüller

Skrub @skrub-data.bsky.social · 23/04/2025

The Skrub TableReport is a lightweight tool that allows to get a rich overview of a table quickly and easily. ✅ Filter columns 🔎 Look at each column's distribution 📊 Get a high level view of the distributions through stats and plots, including correlated columns 🌐 Export the report as html

164

Reposted by David Holzmüller

Gaël Varoquaux @gaelvaroquaux.bsky.social · 23/04/2025

#ICLR2025 Marine Le Morvan presents "Imputation for prediction: beware of diminishing returns": poster Thu 24th arxiv.org/abs/2407.19804 Concludes 6 years of research on prediction with missing values: Imputation is useful but improvements are expensive, while better learners yield easier gains.

1488

Reposted by David Holzmüller

Madelon Hulsebos @madelonhulsebos.bsky.social · 02/04/2025

Excited to share the new monthly Table Representation Learning (TRL) Seminar under the ELLIS Amsterdam TRL research theme! To recur every 2nd Friday. Who: Marine Le Morvan, Inria (in-person) When: Friday 11 April 4-5pm (+drinks) Where: L3.36 Lab42 Science Park / Zoom trl-lab.github.io/trl-seminar/

Details about the seminar talk titled TabICL: A Tabular Foundation Model for In-Context Learning on Large Data by Marine Le Morvan

0113

Reposted by David Holzmüller

Nick Erickson @nickerickson.bsky.social · 25/03/2025

We are excited to announce #FMSD: "1st Workshop on Foundation Models for Structured Data" has been accepted to #ICML 2025! Call for Papers: icml-structured-fm-workshop.github.io

01510

Reposted by David Holzmüller

Evan Peck @peck.phd · 20/11/2024

Trying something new: A 🧵 on a topic I find many students struggle with: "why do their 📊 look more professional than my 📊?" It's *lots* of tiny decisions that aren't the defaults in many libraries, so let's break down 1 simple graph by @jburnmurdoch.bsky.social 🔗 www.ft.com/content/73a1...

921577458

Reposted by David Holzmüller

Multiscale AI @ ICLR 2025 @multiscaleai.bsky.social · 18/03/2025

🚀Continuing the spotlight series with the next @iclr-conf.bsky.social MLMP 2025 Oral presentation! 📝LOGLO-FNO: Efficient Learning of Local and Global Features in Fourier Neural Operators 📷 Join us on April 27 at #ICLR2025! #AI #ML #ICLR #AI4Science

122

David Holzmüller @dholzmueller.bsky.social · 10/03/2025

Practitioners are often sceptical of academic tabular benchmarks, so I am elated to see that our RealMLP model outperformed boosted trees in two 2nd place Kaggle solutions, for a $10,000 forecasting challenge and a research competition on survival analysis.

kaggle.com

Rohlik Sales Forecasting Challenge

Use historical product sales data to predict future sales.

171

David Holzmüller @dholzmueller.bsky.social · 04/03/2025

A new tabular classification benchmark provides another independent evaluation of our RealMLP. RealMLP is the best classical DL model, although some other recent baselines are missing. TabPFN is better on small datasets and boosted trees on larger datasets, though.

120

Reposted by David Holzmüller

Fabian Schaipp @fschaipp.bsky.social · 05/02/2025

Learning rate schedules seem mysterious? Why is the loss going down so fast during cooldown? Turns out that this behaviour can be described with a bound from *convex, nonsmooth* optimization. A short thread on our latest paper 🚞 arxiv.org/abs/2501.18965

arxiv.org

The Surprising Agreement Between Convex Optimization Theory and Learning-Rate Scheduling for Large Model Training

We show that learning-rate schedules for large model training behave surprisingly similar to a performance bound from non-smooth convex optimization theory. We provide a bound for the constant schedul...

2316

Reposted by David Holzmüller

Eugene Berta @eberta.bsky.social · 03/02/2025

Early stopping on validation loss? This leads to suboptimal calibration and refinement errors—but you can do better! With @dholzmueller.bsky.social, Michael I. Jordan, and @bachfrancis.bsky.social, we propose a method that integrates with any model and boosts classification performance across tasks.

4189

David Holzmüller @dholzmueller.bsky.social · 16/01/2025

The first independent evaluation of our RealMLP is here! On a recent 300-dataset benchmark with many baselines, RealMLP takes a shared first place overall. 🔥 Importantly, RealMLP is also relatively CPU-friendly, unlike other SOTA DL models (including TabPFNv2 and TabM). 🧵 1/

Plots from the benchmark of Ye et al. (2024)

191

Reposted by David Holzmüller

Madelon Hulsebos @madelonhulsebos.bsky.social · 07/01/2025

Join us on 27 Feb in Amsterdam for the ELLIS workshop on Representation Learning and Generative Models for Structured Data ✨ sites.google.com/view/rl-and-... Inspiring talks by @eisenjulian.bsky.social, @neuralnoise.com, Frank Hutter, Vaishali Pal, TBC. We welcome extended abstracts until 31 Jan!

2358

Reposted by David Holzmüller

Francis Bach @bachfrancis.bsky.social · 21/12/2024

My book is (at last) out, just in time for Christmas! A blog post to celebrate and present it: francisbach.com/my-book-is-o...

213735

David Holzmüller @dholzmueller.bsky.social · 12/12/2024

I'll present our paper in the afternoon poster session at 4:30pm - 7:30 pm in East Exhibit Hall A-C, poster 3304!

032

David Holzmüller @dholzmueller.bsky.social · 11/12/2024

We wrote a benchmark paper with many practical insights on (the benefits of) active learning for training neural PDE solvers. 🚀 I was happy to be a co-advisor on this project - most of the credit goes to Daniel and Marimuthu.

020

Reposted by David Holzmüller

Moritz Haas @mohaas.bsky.social · 10/12/2024

Stable model scaling with width-independent dynamics? Thrilled to present 2 papers at #NeurIPS 🎉 that study width-scaling in Sharpness Aware Minimization (SAM) (Th 16:30, #2104) and in Mamba (Fr 11, #7110). Our scaling rules stabilize training and transfer optimal hyperparams across scales. 🧵 1/10

1215

David Holzmüller @dholzmueller.bsky.social · 03/12/2024

I'll be at #NeurIPS2024 next week to present this paper (Thu afternoon) as well as a workshop paper on active learning for neural PDE solvers. Let me know if you'd like to chat about tabular data, uncertainty, active learning, etc.!

061

Reposted by David Holzmüller

Michael Kirchhof @mkirchhof.bsky.social · 03/12/2024

Proud to announce our NeurIPS spotlight, which was in the works for over a year now :) We dig into why decomposing aleatoric and epistemic uncertainty is hard, and what this means for the future of uncertainty quantification. 📖 arxiv.org/abs/2402.19460 🧵1/10

37412

David Holzmüller @dholzmueller.bsky.social · 29/11/2024

If you have train+validation data, should you refit on the whole data with the stopping epoch found on the train-validation split? In the quoted paper, we did an experiment including 5-fold ensembles on a 5-fold cross-validation splits (bagging) and with refitting. (short 🧵)

130

Reposted by David Holzmüller

Olivier Grisel @ogrisel.bsky.social · 27/11/2024

I recently shared some of my reflections on how to use probabilistic classifiers for optimal decision-making under uncertainty at @pydataparis.bsky.social 2024. Here is the recording of the presentation: www.youtube.com/watch?v=-gYn...

A high-level summary diagram taken from the slides linked below. It shows the interplay of two main components: a probabilistic model and decision maker or planner.

Probabilistic predictions of an underfitting polynomial classifier on a noisy XOR task and the corresponding under-confident calibration curve.

Probabilistic predictions of an overfitting polynomial classifier and the resulting overconfident calibration curve on the same noisy XOR problem.

Simulation study to show the relative lack of stability of hyperparameter tuning when using hard metrics such as Accuracy or soft yet not probabilistic metrics such as ROC AUC compared to a strictly proper scoring rule such as the log-loss.

15219

David Holzmüller @dholzmueller.bsky.social · 27/11/2024

One thing I learned from this project is that accuracy is a quite noisy metric. With small validation sets (~1K samples), hyperparameter opt. using AUROC instead of accuracy can yield better accuracy on the test set. We also did some experiments on metrics for early stopping. 🧵

230

David Holzmüller @dholzmueller.bsky.social · 25/11/2024

PyTabKit 1.1 is out! - Includes TabM and provides a scikit-learn interface - some baseline NN parameter names are renamed (removed double-underscores) - other small changes, see the readme. github.com/dholzmueller...

1133

Reposted by David Holzmüller

Madelon Hulsebos @madelonhulsebos.bsky.social · 18/11/2024

WIP starterpack w researchers on Table Representation Learning (TRL): all things related to representation learning and generative models for e.g. tables, DBs, spreadsheets! I'll curate but DM/reply w handle+some info welcome! Also follow @trl-research.bsky.social for updates 🤗 go.bsky.app/4SNSMRj

go.bsky.app

Table Representation Learning researchers

Join the conversation

8248

Reposted by David Holzmüller

Michael J. Black @michael-j-black.bsky.social · 20/11/2024

For those who missed this post on the-network-that-is-not-to-be-named, I made public my "secrets" for writing a good CVPR paper (or any scientific paper). I've compiled these tips of many years. It's long but hopefully it helps people write better papers. perceiving-systems.blog/en/post/writ...

perceiving-systems.blog

Writing a good scientific paper

426065

Reposted by David Holzmüller

Skrub @skrub-data.bsky.social · 19/11/2024

@bsky.app is the new cool. So is tabular learning? skrub is a library that eases preprocessing and feature engineering for tabular machine learning. skrub-data.org/stable/ Main features: - scikit-learn compatible - handles Pandas and Polars dataframes - works on heterogeneous types

skrub-data.org

skrub: Less wrangling, more machine learning — skrub

1114

David Holzmüller @dholzmueller.bsky.social · 18/11/2024

Can deep learning finally compete with boosted trees on tabular data? 🌲 In our NeurIPS 2024 paper, we introduce RealMLP, a NN with improvements in all areas and meta-learned default parameters. Some insights about RealMLP and other models on large benchmarks (>200 datasets): 🧵

Paper screenshot and Figure 1 (c) with cumulative ablations for components of RealMLP-TD.

1608

Reposted by David Holzmüller

Marvin Schmitt @marvin-schmitt.com · 17/11/2024

I created a starter pack of scientists in the European Laboratory for Learning and Intelligent Systems (ELLIS) 🇪🇺 Please ping me and I‘ll add you. go.bsky.app/Cihupkk

467727

David Holzmüller @dholzmueller.bsky.social · 16/11/2024

Hi 👋 I'm David Holzmüller, a postdoc at INRIA Paris. I work on supervised (deep) learning, uncertainty quantification, and active learning for tabular data (and some AI4Science). I like to post threads about my research. So please follow me if you like high-SNR "science Twitter" content 🙂

3171

David Holzmüller @dholzmueller.bsky.social · 17/09/2024

The video and slides of my talk are online (link is in the quoted tweet). 📽️ twitter.com/DHolzmueller/status/183…

000

David Holzmüller @dholzmueller.bsky.social · 04/09/2024

Tomorrow I'll be giving a talk on Generalization Theory of Linearized Neural Networks at the MML seminar: www.mis.mpg.de/events/event/general… The (virtual) talk will be live-streamed (5pm CEST = 11am EST = 8am PST) and a...

mis.mpg.de

Event

100

David Holzmüller @dholzmueller.bsky.social · 06/05/2024

I am at ICLR and would love to chat about tabular data, deep active learning, NN uncertainty, etc.! @ViktorZaverkin and I will be presenting a poster for our JMLR paper on deep batch active learning for regression on Tuesday, 10:45 - 12:45, Hall B,...

100

David Holzmüller @dholzmueller.bsky.social · 10/04/2024

Here are two neat kernel theory facts: 📝 (1) Inverse kernel matrices can be studied through norms of minimum-norm interpolators (2) Equivalent kernels have equivalent kernel matrices (details below)

100