iiinova — where your digital self keeps evolving

Pratyush Maini @pratyushmaini.bsky.social · 10/12/2024

Came to #NeurIPS2024 for the research news, but staying for these incredible views. I am presenting some recent works that (I think) significantly advance the discourse on LLM memorization, training data detection; & a study on hallucinations x model collapse in diffusion models.

180

Reposted by Pratyush Maini

Vaishnavh Nagarajan @vaishnavh.bsky.social · 03/12/2024

if you're a PhD student at CMU doing AI/ML, lmk if you want to be added to this starter pack. (I don't belong in this list, but I don't know how to remove myself from this pack 😂) go.bsky.app/9APVxQQ

3143

Reposted by Pratyush Maini

Vishaal Udandarao @vishaalurao.bsky.social · 02/12/2024

🚀New Paper: Active Data Curation Effectively Distills Multimodal Models arxiv.org/abs/2411.18674 Smol models are all the rage these days & knowledge distillation (KD) is key for model compression! We show how data curation can effectively distill to yield SoTA FLOP-efficient {C/Sig}LIPs!! 🧵👇

1236

Reposted by Pratyush Maini

Jia-Bin Huang @jbhuang0604.bsky.social · 01/12/2024

How to drive your research forward? “I tested the idea we discussed last time. Here are some results. It does not work. (… awkward silence)” Such conversations happen so many times when meetings with students. How do we move forward? You need …

19118

Pratyush Maini @pratyushmaini.bsky.social · 27/11/2024

1/Open LLM evals often face data contamination concerns. Private curators (like ScaleAI) have addressed this with private + expert evaluations. We argue that this shift poses new risks including financial incentives & eval bias. w/ @hbxnov.bsky.social 📝: pratyushmaini.github.io/blog/2024/ri... 🧵

162

Reposted by Pratyush Maini

Zachary Lipton @zacharylipton.bsky.social · 26/11/2024

Medically adapted foundation models (think Med-*) turn out to be more hot air than hot stuff. Correcting for fatal flaws in evaluation, the current crop are no better on balance than generic foundation models, even on the very tasks for which benefits are claimed. arxiv.org/abs/2411.04118

arxiv.org

Medical Adaptation of Large Language and Vision-Language Models: Are We Making Progress?

Several recent works seek to develop foundation models specifically for medical applications, adapting general-purpose large language models (LLMs) and vision-language models (VLMs) via continued pret...

825857

Reposted by Pratyush Maini

Anshuman Suri @iamgroot42.bsky.social · 26/11/2024

Temporally shifted data splits in membership inference can be misleading ⚠️ Be cautious when interpreting these benchmarks!

021

Pratyush Maini @pratyushmaini.bsky.social · 26/11/2024

1/6 A lot of us are grappling with peer review these days, but its worst manifestation is when prestigious conference awards overlook critical flaws. Case in point: #EMNLP2024 ’s Best Paper Award. I & @iamgroot42.bsky.social wrote a blog on what went wrong: www.anshumansuri.com/blog/2024/ca... 🧵

170

Pratyush Maini @pratyushmaini.bsky.social · 25/11/2024

1/5 Earlier this year, I joined @datologyai.com to give wings to the data research I had been doing in academia. Today, I am absolutely thrilled to share what we’ve been working on! Techvember Ep 2: How we made the #1 LLM Pre-training Data Recipe. Blog: 👉 tinyurl.com/best-llm-data 🧵

1154

Pratyush Maini @pratyushmaini.bsky.social · 21/11/2024

my new found guilty pleasure is watching the new reasoning models struggle by think-maxxing them with questions from JEE Advanced

160

Pratyush Maini @pratyushmaini.bsky.social · 19/11/2024

pretty excited about tomorrow's class. we will know the winner of our first red-blue team pokemon unlearning challenge. 620 more battles to go ⚔️

140