iiinova — where your digital self keeps evolving

Reposted by Erfan Mirzaei

CSML IIT Lab @pontilgroup.bsky.social · 12/10/2026

📢 We’re hiring! 🧠 Excited about ML foundations & dynamical systems? We have an open Postdoc position in Massimiliano Pontil's CSML unit at @iitalk.bsky.social (Genoa, Italy 🇮🇹) on the ERC "LEO" project! ⏳ Up to 3 years 📅 Deadline: October 31, 2026 Details: 👇 app.ncoreplat.com/jobposition/...

app.ncoreplat.com

Postdoctoral researcher in Machine Learning and Dynamical Systems

Contract and commitment: collaboration contract, for up to three yearsLocation: CHT Erzelli, Genoa, Italy Step into a world of endless possibilities, together let’s leave something for the future! A...

053

Erfan Mirzaei @erfunmirzaei.bsky.social · 13/09/2026

Joining link and schedule: sites.google.com/view/efficie... The key idea is to look at the entire temperature path, not just the final model. We show that the PAC-Bayes complexity term is exactly the area under the mean training-loss curve, separating true learning from noise memorization.

sites.google.com

DLMath&Efficiency

A public ELLIS reading group exploring the interplay between the mathematical foundations of deep learning and the practical challenge of making ML efficient — from optimization theory to hardware-awa...

010

Erfan Mirzaei @erfunmirzaei.bsky.social · 13/09/2026

Overparameterized models fit real data and random data equally well, making final training error a poor predictor of generalization. Tomorrow at 17 CEST at the ELLIS DLMath&Efficiency reading group, I'll present a data-dependent view of this problem using Gibbs and PAC-Bayes. Link to join👇

110

Erfan Mirzaei @erfunmirzaei.bsky.social · 08/07/2026

I couldn't make it in person this time (visa issues), but I'll be online on the ICML virtual page during the session and happy to keep talking after over email. Not at the conference? This thread's for you. 🧵 bsky.app/profile/erfu...

010

Erfan Mirzaei @erfunmirzaei.bsky.social · 08/07/2026

A bit late notice, but if you're curious about the generalization puzzle in overparameterized neural networks, why do some algorithms still generalize even when they can fit random labels? Our poster is up tomorrow: icml.cc/virtual/2026... #ICML2026

icml.cc

ICML Poster Generalization of Gibbs and Langevin Monte Carlo Algorithms in the Interpolation RegimeICML 2026

020

Reposted by Erfan Mirzaei

CSML IIT Lab @pontilgroup.bsky.social · 08/07/2026

ICML 2026 is underway in Seoul, and I'm delighted that our group has four papers on the program this week: 📍 Tuesday, July 7 · Hall A · Poster #4413 Outcome-Aware Spectral Feature Learning for Instrumental Variable Regression icml.cc/virtual/2026...

icml.cc

ICML Poster Outcome-Aware Spectral Feature Learning for Instrumental Variable RegressionICML 2026

121

Reposted by Erfan Mirzaei

CSML IIT Lab @pontilgroup.bsky.social · 17/12/2025

Almost 5 years in the making... "Hyperparameter Optimization in Machine Learning" is finally out! 📘 We designed this monograph to be self-contained, covering: Grid, Random & Quasi-random search, Bayesian & Multi-fidelity optimization, Gradient-based methods, Meta-learning. arxiv.org/abs/2410.22854

0139

Erfan Mirzaei @erfunmirzaei.bsky.social · 28/11/2025

👇 While we wait for the OpenReview drama to settle, here is something that actually solves problems. 😅 A definitive guide to HPO from my lab mates. Don't let your hyperparameters be a mystery (unlike your reviewers). #MachineLearning #HPO

020

Erfan Mirzaei @erfunmirzaei.bsky.social · 14/11/2025

If you’re curious about the intersection of statistical learning theory, sampling-based optimization, generalization in deep learning, and PAC-Bayesian analysis, check out our paper.We’d love to hear your thoughts, feedback, or questions. If you spot interesting connections to your work, let’s chat!

050

Erfan Mirzaei @erfunmirzaei.bsky.social · 14/11/2025

😱 A second, equally striking factor: by applying a single scalar calibration factor computed from the data, the resulting upper bounds become not only tighter for true labels but also better aligned with the test error curve.

120

Erfan Mirzaei @erfunmirzaei.bsky.social · 14/11/2025

🙀 One surprising insight: Generalization in the under-regularized low-temperature regime (β > n) is already signaled by small training errors in the over-regularized high-temperature regime.

110

Erfan Mirzaei @erfunmirzaei.bsky.social · 14/11/2025

Empirical results on MNIST and CIFAR-10 show: 1) Non-trivial upper bounds on test error for both true and random labels 2) Meaningful distinction between structure-rich and structure-poor datasets The figures: Binary classification with FCNNs using SGLD using 8k MNIST images

110

Erfan Mirzaei @erfunmirzaei.bsky.social · 14/11/2025

We show that it can be effectively approximated via Langevin Monte Carlo (LMC) algorithms, such as Stochastic Gradient Langevin Dynamics (SGLD), and crucially, 📎 Our bounds remain stable under this approximation (in both total variation and W₂ distance).

110

Erfan Mirzaei @erfunmirzaei.bsky.social · 14/11/2025

Then comes our first contribution: ✅ We derive high-probability, data-dependent bounds on the test error for hypotheses sampled from the Gibbs posterior (for the first time in the low-temperature regime β > n). Sampling from the Gibbs posterior is, however, typically difficult.

120

Erfan Mirzaei @erfunmirzaei.bsky.social · 14/11/2025

This leads naturally to the Gibbs posterior, which assigns higher probabilities to hypotheses with smaller training errors (exponentially decaying with loss).

110

Erfan Mirzaei @erfunmirzaei.bsky.social · 14/11/2025

To probe this question, we turn to randomized predictors rather than deterministic ones. Here, predictors are sampled from a prescribed probability distribution, allowing us to apply PAC-Bayesian theory to study their generalization properties.

110

Erfan Mirzaei @erfunmirzaei.bsky.social · 14/11/2025

In the figure below from the famous paper, the same model achieves nearly zero training error on both random and true labels. Therefore, the key to generalization must lie within the structure of the data itself. arxiv.org/abs/1611.03530

110

Erfan Mirzaei @erfunmirzaei.bsky.social · 14/11/2025

🧵Thermodynamics Reveals the Generalization in the Interpolation Regime In the realm of overparameterized NNs, one can achieve almost zero training error on any data, even random labels, that yield massive test errors. So, how can we tell when such a model truly generalizes? arxiv.org/abs/2510.06028

arxiv.org

Generalization of Gibbs and Langevin Monte Carlo Algorithms in the Interpolation Regime

The paper provides data-dependent bounds on the test error of the Gibbs algorithm in the overparameterized interpolation regime, where low training errors are also obtained for impossible data, such a...

161

Reposted by Erfan Mirzaei

CSML IIT Lab @pontilgroup.bsky.social · 14/11/2025

📢 Upcoming Talk at Our Lab We’re excited to host Arthur Bizzi from EPFL for a research talk next week! Title: Towards Neural Kolmogorov Equations: Parallelizable SDE Learning with Neural PDEs 🗓 Date: November 19 ⏰ Time: 16:00 CET 📍 Galileo Sala, CHT @iitalk.bsky.social

152

Erfan Mirzaei @erfunmirzaei.bsky.social · 02/05/2025

This paves the way for more data-dependent generalization guarantees in dependent-data settings.

010

Erfan Mirzaei @erfunmirzaei.bsky.social · 02/05/2025

Technique highlights: 🔹 Uses blocking methods 🔹 Captures fast-decaying correlations 🔹 Results in tight O(1/n) bounds when decorrelation is fast Applications: 📊 Covariance operator estimation 🔄 Learning transfer operators for stochastic processes

110

Erfan Mirzaei @erfunmirzaei.bsky.social · 02/05/2025

Our contribution: We propose empirical Bernstein-type concentration bounds for Hilbert space-valued random variables arising from mixing processes. 🧠 Works for both stationary and non-stationary sequences.

110

Erfan Mirzaei @erfunmirzaei.bsky.social · 02/05/2025

Challenge: Standard i.i.d. assumptions fail in many learning tasks, especially those involving trajectory data (e.g., molecular dynamics, climate models). 👉 Temporal dependence and slow mixing make it hard to get sharp generalization bounds.

110

Erfan Mirzaei @erfunmirzaei.bsky.social · 02/05/2025

🚨 Poster at #AISTATS2025 tomorrow! 📍Poster Session 1 #125 We present a new empirical Bernstein inequality for Hilbert space-valued random processes—relevant for dependent, even non-stationary data. w/ Andreas Maurer, @vladimir-slk.bsky.social & M. Pontil 📄 Paper: openreview.net/forum?id=a0E...

130

Reposted by Erfan Mirzaei

CSML IIT Lab @pontilgroup.bsky.social · 15/01/2025

1/ 🚀 Over the past two years, our team, CSML, at IIT, has made significant strides in the data-driven modeling of dynamical systems. Curious about how we use advanced operator-based techniques to tackle real-world challenges? Let’s dive in! 🧵👇

153

Reposted by Erfan Mirzaei

CSML IIT Lab @pontilgroup.bsky.social · 15/01/2025

An inspiring dive into understanding dynamical processes through 'The Operator Way.' A fascinating approach made accessible for everyone—check it out! 👇👀

041

Reposted by Erfan Mirzaei

Riccardo Grazzi @riccardograzzi.bsky.social · 10/12/2024

Excited to present "Unlocking State-Tracking in Linear RNNs Through Negative Eigenvalues" at the M3L workshop at #NeurIPS buff.ly/3BlcD4y If interested, you can attend the presentation the 14th at 15:00, pass at the afternoon poster session, or DM me to discuss :)

buff.ly

Unlocking State-Tracking in Linear RNNs Through Negative Eigenvalues

Linear Recurrent Neural Networks (LRNNs) such as Mamba, RWKV, GLA, mLSTM, and DeltaNet have emerged as efficient alternatives to Transformers in large language modeling, offering linear scaling with…

093

Reposted by Erfan Mirzaei

Pietro Novelli @pienovelli.bsky.social · 12/12/2024

In his book “The Nature of Statistical Learning” V. Vapnik wrote: “When solving a given problem, try to avoid a more general problem as an intermediate step”

183

Erfan Mirzaei @erfunmirzaei.bsky.social · 10/12/2024

Excited to share our lab's amazing contributions at NeurIPS this year! Check out our papers and stay inspired! 🚀📚 #NeurIPS2024

030

Erfan Mirzaei @erfunmirzaei.bsky.social · 27/11/2024

Hi Gaspard. I wonder what you are currently working on in regard to sequence models and world models. Since I have similar interests as you, and in the lab, we had worked on the intersection of the topics (bsky.app/profile/marc...).

120

Reposted by Erfan Mirzaei

ELLIS @ellis.eu · 21/11/2024

Hi 👋 We're glad to be here on @bsky.app and looking forward to engaging in this community. But first, learn a little more about us... #ELLISforEurope #AI #ML #CrossBorderCollab #PhD

313120