iiinova — where your digital self keeps evolving

Johannes Zenn @johanneszenn.bsky.social · 29/08/2025

Reposted by Sebastian Bordt

A new recording of our FridayTalks@Tübingen series is online! How much can we forget about Data Contamination? by @sbordt.bsky.social Watch here: youtu.be/T9Y5-rngOLg

youtu.be

How much can we forget about Data Contamination? - [Sebastian Bordt]

YouTube video by Friday Talks Tübingen

121 0

Sebastian Bordt @sbordt.bsky.social · 23/07/2025

Thanks for your comments. I don't think that neural networks are just a form of memory (though they can store a large number of memories). For example, transformers with unbounded steps are Turing-complete direct.mit.edu/tacl/article...

direct.mit.edu

What Formal Languages Can Transformers Express? A Survey

Abstract. As transformers have gained prominence in natural language processing, some researchers have investigated theoretically what problems they can and cannot solve, by treating problems as forma...

010 0

Sebastian Bordt @sbordt.bsky.social · 20/07/2025

I see the point of the original post, but I think it's also important to keep in mind this other aspect.

Sebastian Bordt @sbordt.bsky.social · 20/07/2025

www.inference.vc/we-may-be-su...

171 0

Reposted by Sebastian Bordt

Grace @gracekind.net · 19/07/2025

The stochastic parrot is now an IMO gold medalist parrot

2557 0

Sebastian Bordt @sbordt.bsky.social · 14/07/2025

Wednesday: Position: Rethinking Explainable Machine Learning as Applied Statistics icml.cc/virtual/2025...

010 0

Sebastian Bordt @sbordt.bsky.social · 14/07/2025

I'm at #ICML in Vancouver this week, hit me up if you want to chat about pre-training experiments or explainable machine learning. You can find me at these posters: Tuesday: How Much Can We Forget about Data Contamination? icml.cc/virtual/2025...

111 0

Sebastian Bordt @sbordt.bsky.social · 11/07/2025

Great to hear that you like it, and thank you for the feedback! I agree that stakeholders are important, although you are not going to find much about it in this paper. We might argue, though, that similar aspects with stakeholders arise in data science with large datasets, hence the analogy :)

000 0

Reposted by Sebastian Bordt

Ulrike Luxburg @ulrikeluxburg.bsky.social · 11/07/2025

Our #ICML position paper: #XAI is similar to applied statistics: it uses summary statistics in an attempt to answer real world questions. But authors need to state how concretely (!) their XAI statistics contributes to answer which concrete (!) question! arxiv.org/abs/2402.02870

062 0

There are many more interesting aspects to this, so take a look at our paper! arxiv.org/abs/2402.02870 We would also be happy for questions and comments on why we got it completely wrong.😊 If you are at ICML, I will present this paper on Wed 16 Jul 4:30 in the East Exhibition Hall A-B #E-501.📍

000 0

We think the literature on explainable machine learning can learn a lot from looking at these papers!📚

As I learned from our helpful ICML reviewers, there is a lot of existing research at the intersection of machine learning and statistics that takes the matter of interpretation quite seriously.

100 0

In this framework, another way to formulate the initial problems is: For many popular explanation algorithms, it is not clear whether they have an interpretation.

100 0

Having an interpretation means that the explanation formalizes an intuitive human concept, which is a fancy philosophical way of saying that it is clear what aspect of the function the explanation describes.🧠

100 0

In addition, the way to develop explanations that are useful "in the world" is to develop explanations that have an interpretation.

100 0

This has several important implications. Most importantly, explainable machine learning has often been trying to reinvent the wheel when we already have a robust framework for discussing complex objects in the light of pressing real-world questions.

It took us a while to recognize it, but once you see it, you can't unsee it: Explainable machine learning is applied statistics for learned functions.✨

100 0

Concretely, researchers in applied statistics study complex datasets by mapping their most important properties into low-dimensional structures. Now think: Machine learning model ~ Large dataset Explanation algorithm ~ Summary statistics, visualization

120 0

Here comes our key realization: This question has occurred in other disciplines before, specifically in applied statistics research.

So, how can we seriously discuss whether an explanation algorithm can be used to answer relevant questions about our trained model or the world?🌍

100 0

I have actually encountered this point in my own research before, where we did a detailed mathematical analysis of SHAP, but all the math could not reveal the right way to use the explanations in practice (arxiv.org/abs/2209.040...

100 0

And then they never make clear how the technical properties of the algorithm tie back to the grander goals of explainability research (meaning: how do we actually use the algorithm in practice?)🎯

The problem: Papers about explanation algorithms often start with grand promises, but then quickly dive into technical details.

100 0

During the last couple of years, we have read a lot of papers on explainability and often felt that something was fundamentally missing🤔 This led us to write a position paper (accepted at #ICML2025) that attempts to identify the problem and to propose a solution. arxiv.org/abs/2402.02870 👇🧵

1125 0

joint work with Suraj Srinivas, Valentyn Boreiko, and @ulrikeluxburg.bsky.social Link to paper: arxiv.org/pdf/2410.03249 Link to code: github.com/tml-tuebinge... We would be happy for feedback or questions about this work!

000 0

Key takeaways: ✅ You can't talk about data contamination without considering the scale of the model and data ✅ Many LLM training setups are fairly robust to a few times of contamination ✅ Weight decay provides a useful heuristic for forgetting in large-scale training

100 0

🔧What are the reasons for the forgetting? We highlight one important factor: The weight decay parameter of AdamW. Concretely, we show that forgetting data contamination always occurs at least as fast as the decay of past gradients in AdamW.

100 0

If you carefully look at the previous plot, you will see that forgetting exhibits a nice scaling behavior.🔬

100 0

🧠 The mechanism? Forgetting dynamics! As the model contines training, texts that were seen earlier become less and less important. As we illustrate with OLMo-7B, this effect persists in fairly large models.

100 0

💡For a model trained at 15x Chinchilla, we do not find significant benchmark overfitting even if the contaminated data is duplicated 12 times within the training data.

100 0

🚀But modern LLMs are not Chinchilla-optimal, they are trained on significantly more tokens. And as the overall size of the training data increases, the impact of contamination starts to decrease.

100 0

⚠️ Under Chinchilla scaling (where the data is 20x model parameters), a single time of contamination can lead to overfitting of a few percentage points.

100 0

Key insight: The impact of contamination depends on three factors: Model size📏 Training data volume📚 Contamination frequency🔄 The interplay between these determines whether a model actually overfits on downstream evaluations.

100 0

Sebastian Bordt @sbordt.bsky.social · 03/07/2025

Have you ever wondered whether a few times of data contamination really lead to benchmark overfitting?🤔 Then our latest #ICML paper about the effect of data contamination on LLM evals might be for you!🚀 Paper: arxiv.org/abs/2410.03249 👇🧵

1121 0

I like this decision-theoretic perspective :) We have also been frustrated for a while with the state of explainable machine learning and summarized our thoughts in this position paper: arxiv.org/abs/2402.02870. Perhaps you find it interesting