iiinova — where your digital self keeps evolving

Raman Dutt @ramandutt4.bsky.social · 22/02/2025

This de identification token is consistently present throughout the dataset and contributes nothing towards improving the image quality. This points towards a major flaw in the dataset given MIMIC is one of the most significant medical datasets for T2I generation. 💔

000

Raman Dutt @ramandutt4.bsky.social · 22/02/2025

It turns out that this de identification token (“___”) holds the most significant contribution towards memorizing training images. In other words, steps taken to protect patient information are in fact posing a threat to it.

000

Raman Dutt @ramandutt4.bsky.social · 22/02/2025

MIMIC dataset contains pairs of images and corresponding text reports. These are raw reports describing the images. In the dataset, the sensitive patient information is hidden or de identified. This is done by replacing it with three underscores (“___”).

200

Raman Dutt @ramandutt4.bsky.social · 22/02/2025

Are you working on Text-to-Image generation of Chest X-Rays using the MIMIC dataset? Here is something I found over the last weekend. 🧵 Observations documented in this preprint - arxiv.org/abs/2502.07516

arxiv.org

100

Raman Dutt @ramandutt4.bsky.social · 07/12/2024

Hello all, What’s the best tool to make nice figures for academic AI papers?

180

Raman Dutt @ramandutt4.bsky.social · 06/12/2024

Not an ML-related post but I am just as happy to share this

080

Raman Dutt @ramandutt4.bsky.social · 05/12/2024

1000 followers on Bluesky already that’s crazy

090

Raman Dutt @ramandutt4.bsky.social · 27/11/2024

A new starter pack for Medical AI researchers! go.bsky.app/PddA2uy

051

Raman Dutt @ramandutt4.bsky.social · 26/11/2024

Sharing with people who might find this relevant - go.bsky.app/r5eVvT, go.bsky.app/PJKJ8vK, bsky.app/profile/berk...

010

Raman Dutt @ramandutt4.bsky.social · 26/11/2024

Hence, we devise MemControl - a framework that searches for the optimal parameters to be fine-tuned to: (1) Improve image generation quality (2) Reduce Memorization! MemControl leads to optimal model capacity that should be used during fine-tuning: Not more, not less!

000

Raman Dutt @ramandutt4.bsky.social · 26/11/2024

We also found that fine-tuning different subsets of parameters in a diffusion model can affect generative quality and memorization differently! Each marker in the figure is a diffusion model finetuned on the same data but with different parameter subset. Full FT (green) leads to high memorization!

110

Raman Dutt @ramandutt4.bsky.social · 26/11/2024

We provide empirical proof that reducing the model capacity (by fine-tuning fewer parameters) can lead to reduced memorization! Q. How to fine-tune with fewer parameters? 🤔 A. Parameter-Efficient Fine-Tuning (PEFT) ✨

100

Raman Dutt @ramandutt4.bsky.social · 26/11/2024

The conventional way of fine-tuning models (full fine-tuning) can lead to replication of artifacts in X-Rays that can further lead to leakage of patient information, thus endangering patient privacy. Artifact replication is shown in red boxes.

Figure showing how conventional fine-tuning methods can lead to replication of artifacts (red boxes)

100

Raman Dutt @ramandutt4.bsky.social · 26/11/2024

Delighted to share our work "𝐌𝐞𝐦𝐂𝐨𝐧𝐭𝐫𝐨𝐥" now accepted at 𝐖𝐀𝐂𝐕 '𝟐𝟓. We show strong results for medical image generation and also establish an initial benchmark for generative quality and memorization of synthetic chest x-rays! Paper: arxiv.org/abs/2405.19458 Code: github.com/Raman1121/Di... More👇

github.com

GitHub - Raman1121/Diffusion_Memorization_HPO: A framework to reduce memorization in text-to-image diffusion models using HPO

A framework to reduce memorization in text-to-image diffusion models using HPO - Raman1121/Diffusion_Memorization_HPO

160

Raman Dutt @ramandutt4.bsky.social · 26/11/2024

MIDL Conference has joined BlueSky! bsky.app/profile/midl...

bsky.app

020

Raman Dutt @ramandutt4.bsky.social · 25/11/2024

I would love to be added 😂

000

Raman Dutt @ramandutt4.bsky.social · 25/11/2024

Again, very sorry to hear about what you are going through. Advertising here is a great idea. I personally got some good advice about a condition I was going through. Wish I could be more helpful though. Wishing you the best!

010

Raman Dutt @ramandutt4.bsky.social · 25/11/2024

So sorry to hear about this @ian-goodfellow.bsky.social . Do you think any of this might be related to prolonged headphone usage in addition to many other factors? Or if that exacerbates the condition?

120

Raman Dutt @ramandutt4.bsky.social · 25/11/2024

For my fellow medical AI researchers, here is a starter pack - go.bsky.app/r5eVvT

152

Raman Dutt @ramandutt4.bsky.social · 25/11/2024

I would consider myself slightly cracked haha. Would love to be added!

100

Raman Dutt @ramandutt4.bsky.social · 25/11/2024

A starter pack I would highly recommend (not biased at all 😉)

150

Raman Dutt @ramandutt4.bsky.social · 25/11/2024

Would love to be added! Currently doing a PhD in Biomedical AI

120

Raman Dutt @ramandutt4.bsky.social · 24/11/2024

Omg BlueSky migration success story continues :’) @chriswolfvision.bsky.social

020

Raman Dutt @ramandutt4.bsky.social · 24/11/2024

Haha no. Gotta start from scratch

110

Raman Dutt @ramandutt4.bsky.social · 24/11/2024

Migration to BlueSky successful! Thank you @mmbronstein.bsky.social 🙏

1151

Raman Dutt @ramandutt4.bsky.social · 24/11/2024

Would love to be added! Currently starting the final year of my PhD at the University of Edinburgh!

010

Raman Dutt @ramandutt4.bsky.social · 24/11/2024

Computer Vision starter pack!

010

Raman Dutt @ramandutt4.bsky.social · 24/11/2024

I would love to be added! Currently at Huawei and Turing Institute working on Autoregressive Multimodal Generation.

210

Reposted by Raman Dutt

Andrei Bursuc @abursuc.bsky.social · 22/11/2024

The return of the Autoregressive Image Model: AIMv2 now going multimodal. Excellent work by @alaaelnouby.bsky.social & team with code and checkpoints already up: arxiv.org/abs/2411.14402

1468

Raman Dutt @ramandutt4.bsky.social · 21/11/2024

Here is a starter pack of researchers from the University of Edinburgh working on all areas of AI! go.bsky.app/KRNDkN7

041

Raman Dutt @ramandutt4.bsky.social · 21/11/2024

Here is a well-curated list of researchers working on image/ video generation

000

Raman Dutt @ramandutt4.bsky.social · 21/11/2024

Would love to be added to the list! Here is my recent work on reducing memorization in diffusion models, inspired by yours :’) Recently accepted at WACV ‘25 Paper - arxiv.org/pdf/2405.19458

arxiv.org

100