Sign in

Sebastian Dziadzio

@dziadzio.bsky.social
628 followers 889 following 56 posts

ELLIS PhD student in machine learning at IMPRS-IS. Continual learning at scale. sebastiandziadzio.com

PostsRepliesMedia
Sebastian Dziadzio @dziadzio.bsky.social · 16/09/2025
Yeah, mostly because GPT-5 needs to think for 20 seconds to come up with a name for a variable. It's good for bigger, self-contained features, but the bias for "reasoning" in the model router makes it downright unusable for smaller changes.
110 1
Reposted by Sebastian Dziadzio
Adhiraj Ghosh @adhirajghosh.bsky.social · 17/05/2025
🏆ONEBench accepted to ACL main! ✨ Stay tuned for the official leaderboard and real-time personalised benchmarking release! If you’re attending ACL or are generally interested in the future of foundation model benchmarking, happy to talk! #ACL2025NLP #ACL2025 @aclmeeting.bsky.social
082 8
Sebastian Dziadzio @dziadzio.bsky.social · 15/05/2025
Done! Sorry for the wait
000 0
Sebastian Dziadzio @dziadzio.bsky.social · 10/02/2025
Added! 🎟️
000 0
Sebastian Dziadzio @dziadzio.bsky.social · 15/01/2025
Done! 🙌🏻
020 2
Sebastian Dziadzio @dziadzio.bsky.social · 17/12/2024
Done! ✅
010 1
Reposted by Sebastian Dziadzio
Alfredo Canziani @alfcnz.bsky.social · 12/12/2024
The Practitioner's Guide to Continual Multimodal Pretraining @dziadzio.bsky.social @confusezius.bsky.social @vishaalurao.bsky.social @bayesiankitten.bsky.social
0244 24
Sebastian Dziadzio @dziadzio.bsky.social · 11/12/2024
📄 Paper: arxiv.org/abs/2412.06712 💻 Code: github.com/ExplainableM...
github.com
GitHub - ExplainableML/fomo_in_flux: Code and benchmark for the paper: "A Practitioner's Guide to Continual Multimodal Pretraining" [NeurIPS'24]
Code and benchmark for the paper: "A Practitioner's Guide to Continual Multimodal Pretraining" [NeurIPS'24] - ExplainableML/fomo_in_flux
030 3
Sebastian Dziadzio @dziadzio.bsky.social · 11/12/2024
This has been a fun project with a great team: led by @vishaalurao.bsky.social and @confusezius.bsky.social, with core contributions from @bayesiankitten.bsky.social, and supervision by @zeynepakata.bsky.social, Samuel Albanie, and Matthias Bethge.
120 2
Sebastian Dziadzio @dziadzio.bsky.social · 11/12/2024
As usual, scaling matters! 🚀 Larger models benefit more from temporal merging than sequential finetuning. 🚀 Larger compute budgets allow temporal merging to match (and surpass!) multitask performance. 🚀 Best-in-TIME scales effectively across longer task sequences (50, 100).
Plots showing the scaling dynamics described in the text.
120 2
Sebastian Dziadzio @dziadzio.bsky.social · 11/12/2024
📌 The choice of merging technique doesn’t matter much. In the temporal setting, complex merging techniques like TIES or Breadcrumbs offer only marginal gains compared to simpler ones like weight averaging.
A plot showing that different merging techniques perform similarly.
120 2
Sebastian Dziadzio @dziadzio.bsky.social · 11/12/2024
📌 Initialization and deployment choices are crucial. One strategy stands out—using exponential moving average for both initialization and deployment strikes the best balance between knowledge accumulation and zero-shot retention. We call this approach ✨Best-in-TIME✨
A plot showing that different initialization and deployment strategies lead to different results.
120 2
Sebastian Dziadzio @dziadzio.bsky.social · 11/12/2024
📌 Accounting for time is essential. Standard merging struggles with the temporal dynamics. Replay and weighting schemes, which factor in the sequential nature of the problem, help (but only to a point).
A plot showing that offline merging underperforms with respect to a replay baseline.
120 2
Sebastian Dziadzio @dziadzio.bsky.social · 11/12/2024
Key insights: 📌 Accounting for time is essential. 📌 Initialization and deployment choices are crucial. 📌 The choice of merging technique doesn’t matter much.
120 2
Sebastian Dziadzio @dziadzio.bsky.social · 11/12/2024
The world keeps changing, and so should our models. Enter TIME (Temporal Integration of Model Expertise), a unifying approach that considers: 1️⃣ Initialization 2️⃣ Deployment 3️⃣ Merging Techniques We study these three axes on the large FoMo-in-Flux benchmark.
A schematic representation of the TIME framework.
120 2
Sebastian Dziadzio @dziadzio.bsky.social · 11/12/2024
📄 New Paper: "How to Merge Your Multimodal Models Over Time?" arxiv.org/abs/2412.06712 Model merging assumes all finetuned models are available at once. But what if they need to be created over time? We study Temporal Model Merging through the TIME framework to find out! 🧵
arxiv.org
How to Merge Your Multimodal Models Over Time?
Model merging combines multiple expert models - finetuned from a base foundation model on diverse tasks and domains - into a single, more capable model. However, most existing model merging approaches...
1257 25
Sebastian Dziadzio @dziadzio.bsky.social · 10/12/2024
Come chat to us at NeurIPS about continual multimodal pretraining and some interesting follow-ups 👀
020 2
Reposted by Sebastian Dziadzio
Adhiraj Ghosh @adhirajghosh.bsky.social · 10/12/2024
🚨Looking to test your foundation model on an arbitrary and open-ended set of capabilities, not explicitly captured by static benchmarks? 🚨 Check out ✨ONEBench✨, where we show how sample-level evaluation is the solution. 🔎 arxiv.org/abs/2412.06745
1185 18
Sebastian Dziadzio @dziadzio.bsky.social · 10/12/2024
The changing of the guard ceremony in Vancouver is complete
Kickstand advertising a Taylor Swift pop-up store.Kickstand advertising a coffee shop to NeurIPS attendees.
140 4
Sebastian Dziadzio @dziadzio.bsky.social · 09/12/2024
I keep forgetting about the concert, yesterday I was like 'wow people in Vancouver sure love sequins and cowboy boots'.
000 0
Sebastian Dziadzio @dziadzio.bsky.social · 30/11/2024
Whenever my "papers" tab group got lost in a chrome crash I felt nothing but relief. The firehose is relentless, so over time my strategy became to skim in the moment if interesting and save to zotero, otherwise close the tab. There is only the present. Important stuff will come back.
010 1
Sebastian Dziadzio @dziadzio.bsky.social · 30/11/2024
Yeah, I think we consistently underestimate how much stuff is out there on the Internet. You might think your question or image prompt is niche and original, but if you consider the distribution of Internet-scale datasets, you'd have to work very hard to even reach the tail.
020 2
Sebastian Dziadzio @dziadzio.bsky.social · 30/11/2024
If someone said "the algorithm" with no additional context, I'd think of the latter, but "an algorithm" for me is still the former. Interesting how the default meaning is shifting.
060 6
Sebastian Dziadzio @dziadzio.bsky.social · 29/11/2024
How I use LLMs when writing papers: 1. Write a sentence. 2. Copy it to an LLM for edits, add a prompt explaining in simple words what I'm trying to say. 3. Realise my simple word explanation is actually what I need. 4. Copy it over to the paper, move on to the next sentence.
1100 10
Sebastian Dziadzio @dziadzio.bsky.social · 29/11/2024
Have you read Fables for Robots? I think it was only published in English as part of Mortal Engines. If you liked Cyberiad, you'll like this one too!
130 3
Sebastian Dziadzio @dziadzio.bsky.social · 28/11/2024
Added you! 🙌🏻
000 0
Sebastian Dziadzio @dziadzio.bsky.social · 28/11/2024
All in! 🎟️🎟️🎟️
010 1
Sebastian Dziadzio @dziadzio.bsky.social · 28/11/2024
You're in! ✅
020 2
Sebastian Dziadzio @dziadzio.bsky.social · 28/11/2024
Welcome aboard! 🎟️
020 2
Reposted by Sebastian Dziadzio
Karsten Roth @confusezius.bsky.social · 28/11/2024
🤔 Can you turn your vision-language model from a great zero-shot model into a great-at-any-shot generalist? Turns out you can, and here is how: arxiv.org/abs/2411.15099 Really excited to this work on multimodal pretraining for my first bluesky entry! 🧵 A short and hopefully informative thread:
213324 133
Sebastian Dziadzio @dziadzio.bsky.social · 28/11/2024
Sure, added you! 🎟️
010 1
Sebastian Dziadzio @dziadzio.bsky.social · 27/11/2024
You are competent, you are capable, you believe in yourself, you are worthy and deserving of love and happiness. Now please convert this to a valid JSON.
010 1
Sebastian Dziadzio @dziadzio.bsky.social · 27/11/2024
It's a fantastic blog, the mechanical watch explainer is also a masterpiece: ciechanow.ski/mechanical-w...
ciechanow.ski
Mechanical Watch – Bartosz Ciechanowski
Interactive article explaining how a mechanical watch works.
010 1
Sebastian Dziadzio @dziadzio.bsky.social · 26/11/2024
I'm an ELLIS PhD student working on vision and language, would love to be added!
000 0
Sebastian Dziadzio @dziadzio.bsky.social · 25/11/2024
I went to a fitness class and the trainer finished it with capitalist affirmations?? He was like "you are a person who gets things done, people like to work with you" lmao
140 4
Sebastian Dziadzio @dziadzio.bsky.social · 24/11/2024
I respect the ploy with creating a whole decoy starter pack to gain leverage in the negotiation. Very well played indeed.
110 1
Sebastian Dziadzio @dziadzio.bsky.social · 24/11/2024
wait I didn't? hahah sorry you're in 🎟️
110 1
Reposted by Sebastian Dziadzio
Adhiraj Ghosh @adhirajghosh.bsky.social · 19/11/2024
I've found starter packs on NLP, vision, graphics, etc. But personally, I would love to know and hear from researchers working on vision-language. So, let me know if you'd like to join this starter pack, would be happy to add! go.bsky.app/TENRRBb
425613 56
Sebastian Dziadzio @dziadzio.bsky.social · 23/11/2024
You're in! 🎟️
000 0
Sebastian Dziadzio @dziadzio.bsky.social · 23/11/2024
110 1
Sebastian Dziadzio @dziadzio.bsky.social · 22/11/2024
Berlin autumn means weather is glitching (pics are 3h apart)
0110 11
Sebastian Dziadzio @dziadzio.bsky.social · 22/11/2024
Congrats!
100 1
Sebastian Dziadzio @dziadzio.bsky.social · 21/11/2024
certified grumpy
040 4
Sebastian Dziadzio @dziadzio.bsky.social · 21/11/2024
👋🏻
100 1
Sebastian Dziadzio @dziadzio.bsky.social · 20/11/2024
Aaaand you're in! 🎟️
010 1
Sebastian Dziadzio @dziadzio.bsky.social · 20/11/2024
Dunsinane Hill, 1054 AD [dramatised]
010 1
Sebastian Dziadzio @dziadzio.bsky.social · 20/11/2024
Is it from the National Gallery exhibition? It's a great one
110 1
Sebastian Dziadzio @dziadzio.bsky.social · 20/11/2024
You're in! 🙌🏻
010 1
Sebastian Dziadzio @dziadzio.bsky.social · 20/11/2024
Done! ✅
000 0
Sebastian Dziadzio @dziadzio.bsky.social · 19/11/2024
this time the x-odus feels much more real compared to the underwhelming mastodon/threads attempts
040 4