iiinova — where your digital self keeps evolving

Reposted by Amartya Sanyal

Omri Ben-Dov @omribendov.bsky.social · 22/08/2025

In our new work we ask: Can end-users make a platform’s ML models fairer? Firm-side fair learning often reduces accuracy, discouraging firms from using it. But if a platform relies on user data, can minority users collectively change the data to induce fairness? (1/4)

A classifier has an error of 0.15 and unfairness violation of 0.13, while the same classifier, on data with 6 relabeled samples, have the the same error but 0.03 fairness violation.

112

Reposted by Amartya Sanyal

Differential Privacy Papers @dppapers.bsky.social · 15/08/2025

An Iterative Algorithm for Differentially Private $k$-PCA with Adaptive Noise Johanna Düngler, Amartya Sanyal arxiv.org/abs/2508.10879

$An Iterative Algorithm for Differentially Private $k$-PCA with Adaptive Noise Johanna Düngler, Amartya Sanyal http://arxiv.org/abs/2508.10879 Given $n$ i.i.d. random matrices $A_i \in \mathbb{R}^{d \times d}$ that share a common expectation $\Sigma$, the objective of Differentially Private Stochastic PCA is to identify a subspace of dimension $k$ that captures the largest variance directions of $\Sigma$, while preserving differential privacy (DP) of each individual $A_i$. Existing methods either (i) require the sample size $n$ to scale super-linearly with dimension $d$, even under Gaussian assumptions on the $A_i$, or (ii) introduce excessive noise for DP even when the intrinsic randomness within $A_i$ is small. Liu et al. (2022a) addressed these issues for sub-Gaussian data but only for estimating the top eigenvector ($k=1$) using their algorithm DP-PCA. We propose the first algorithm capable of estimating the top $k$ eigenvectors for arbitrary $k \leq d$, whilst overcoming both limitations above. For $k=1$ our algorithm matches the utility guarantees of DP-PCA, achieving near-optimal statistical error even when $n = \tilde{\!O}(d)$. We further provide a lower bound for general $k > 1$, matching our upper bound up to a factor of $k$, and experimentally demonstrate the advantages of our algorithm over comparable baselines.$

031

Amartya Sanyal @amartyasanyal.bsky.social · 15/05/2025

🚨New Paper: Online Learning and Unlearning 🚨 We look at learning and unlearning in the online setting where both learning and unlearning requests arrive continuously over time. Lead by @yaxihu.bsky.social and joint work with Bernhard Schölkopf arxiv.org/abs/2505.08557

171

Reposted by Amartya Sanyal

UKP Lab @ukplab.bsky.social · 23/04/2025

𝗖𝗮𝗻 𝘄𝗲 𝗮𝗹𝗶𝗴𝗻 𝗟𝗟𝗠𝘀 𝘁𝗼𝘄𝗮𝗿𝗱𝘀 𝗮 𝗱𝗲𝘀𝗶𝗿𝗲𝗱 𝗯𝗲𝗵𝗮𝘃𝗶𝗼𝘂𝗿 𝘄𝗵𝗶𝗹𝗲 𝗺𝗮𝗶𝗻𝘁𝗮𝗶𝗻𝗶𝗻𝗴 𝗱𝗶𝗳𝗳𝗲𝗿𝗲𝗻𝘁𝗶𝗮𝗹 𝗽𝗿𝗶𝘃𝗮𝗰𝘆 𝗴𝘂𝗮𝗿𝗮𝗻𝘁𝗲𝗲𝘀? We answer this in our #ICLR2025 paper. Tl;dr: We propose, evaluate and audit a novel differentially private activation steering algorithm for aligning LLMs. (1/🧵)

141

Amartya Sanyal @amartyasanyal.bsky.social · 27/03/2025

Open postdoc position in learning theory/ privacy/ robustness/ unlearning or any related topics with me and others in University of Copenhagen, Denmark. If you think you would be a good candidate, send me an email amartya18x.github.io/hiring/ #postdoc

amartya18x.github.io

Vacancies

040

Reposted by Amartya Sanyal

Konrad Rieck 🌈 @rieck.mlsec.org · 03/03/2025

No plans for April 9–11 yet? — Why not spend an amazing week in beautiful Copenhagen 🇩🇰, exploring cutting-edge research on trustworthy machine learning. Join us at SaTML 2025, the premier conference on AI security, AI privacy, and AI fairness! 👉 satml.org/attend @satml.org

IEEE Conference on Secure and Trustworthy Machine Learning (SaTML), April 9-11, 2025, Copenhagen.

074

Amartya Sanyal @amartyasanyal.bsky.social · 25/02/2025

Very shortly at @realaaai.bsky.social, @alext2.bsky.social and I will be giving a Tutorial on the impact of Quality and availability of labels and data for Privacy, Fairness, and Robustness of ML algorithms See here amartya18x.github.io/files/Tutori... @ucph.bsky.social

041

Reposted by Amartya Sanyal

Pioneer Centre for AI @aicentre.dk · 22/02/2025

3rd IEEE Conference on Secure and Trustworthy Machine Learning University of Copenhagen, Denmark, April 9-11, 2025 - registration is open. satml.org @amartyasanyal.bsky.social

satml.org

IEEE SaTML

IEEE Conference on Secure and Trustworthy Machine Learning

061

Reposted by Amartya Sanyal

Differential Privacy Papers @dppapers.bsky.social · 31/01/2025

Differentially Private Steering for Large Language Model Alignment Anmol Goel, Yaxi Hu, Iryna Gurevych, Amartya Sanyal arxiv.org/abs/2501.18532

$Differentially Private Steering for Large Language Model Alignment Anmol Goel, Yaxi Hu, Iryna Gurevych, Amartya Sanyal http://arxiv.org/abs/2501.18532 Aligning Large Language Models (LLMs) with human values and away from undesirable behaviors (such as hallucination) has become increasingly important. Recently, steering LLMs towards a desired behavior via activation editing has emerged as an effective method to mitigate harmful generations at inference-time. Activation editing modifies LLM representations by preserving information from positive demonstrations (e.g., truthful) and minimising information from negative demonstrations (e.g., hallucinations). When these demonstrations come from a private dataset, the aligned LLM may leak private information contained in those private samples. In this work, we present the first study of aligning LLM behavior with private datasets. Our work proposes the \textit{\underline{P}rivate \underline{S}teering for LLM \underline{A}lignment (PSA)} algorithm to edit LLM activations with differential privacy (DP) guarantees. We conduct extensive experiments on seven different benchmarks with open-source LLMs of different sizes (0.5B to 7B) and model families (LlaMa, Qwen, Mistral and Gemma). Our results show that PSA achieves DP guarantees for LLM alignment with minimal loss in performance, including alignment metrics, open-ended text generation quality, and general-purpose reasoning. We also develop the first Membership Inference Attack (MIA) for evaluating and auditing the empirical privacy for the problem of LLM steering via activation editing. Our attack is tailored for activation editing and relies solely on the generated texts without their associated probabilities. Our experiments support the theoretical guarantees by showing improved guarantees for our \textit{PSA} algorithm compared to several existing non-private techniques.$

021

Reposted by Amartya Sanyal

UKP Lab @ukplab.bsky.social · 27/01/2025

#ICLR »Differentially Private Steering for Large Language Model Alignment« by @anmolgoel.bsky.social, Yaxi Hu, Iryna Gurevych (@igurevych.bsky.social) & Amartya Sanyal (@amartyasanyal.bsky.social) (2/🧵)

163

Amartya Sanyal @amartyasanyal.bsky.social · 24/01/2025

I was very lucky and happy to be awarded the Villum Young Investigator grant yesterday villumfonden.dk/en/news/mill... Looking forward to the resulting research in unlearning, privacy, and online learning supported by the Villum foundation. (Hiring motivated PhDs and postdocs, especially postdocs)

villumfonden.dk

Millions in funding for young researchers

Nineteen promising researchers working within the technical and natural sciences have received funding of DKK 150 million for their research projects.

4170

Reposted by Amartya Sanyal

Mahdi Haghifam @mahdihaghifam.bsky.social · 11/12/2024

I’ll be at #NeurIPS2024 this week! Looking forward to presenting my joint work with Thomas Steinke(@stein.ke) and Jon Ullman(@thejonullman.bsky.social) NeurIPS page with video: neurips.cc/virtual/2024... Link to arxiv: arxiv.org/abs/2406.07407

0103

Reposted by Amartya Sanyal

Daniil Dmitriev @ddmitriev.bsky.social · 10/12/2024

Excited to present at #NeurIPS2024 our work on robust mixture learning! How hard is mixture learning when (a lot of) outliers are present? We show that it's easier than it seems! Join us at the poster session (Wed, 16:30 PT, West Ballroom A-D #5710).

132

Reposted by Amartya Sanyal

Rasmus Pagh @rasmuspagh.net · 03/12/2024

Graduating with a PhD related to privacy and robustness in machine learning? Apply to this post-doc opening by @amartyasanyal.bsky.social: employment.ku.dk/faculty/?sho...

employment.ku.dk

Postdoc in Privacy and Robustness of Machine Learning Algorithms

021

Amartya Sanyal @amartyasanyal.bsky.social · 02/12/2024

Two weeks remaining to apply to this position. I'll also be at NeurIPS, if you want to chat you can DM or email me. :)

042

Amartya Sanyal @amartyasanyal.bsky.social · 27/11/2024

Three long research meetings throughout the day with four brilliant collaborators. It was a good day.

040

Reposted by Amartya Sanyal

Clément Canonne @ccanonne.github.io · 25/11/2024

On the TCS job market: Hao Wu! Hao Wu's research interests focus on both the theoretical and practical aspects of differentially private data analysis. He is actively pursuing opportunities in academia and industry. 1/2 #TCSSky #AcademicJobMarket

152

Reposted by Amartya Sanyal

Nando Fioretto @nandofioretto.bsky.social · 23/11/2024

🆘Help needed! Are you working on Privacy (from a Technical (e.g., Differential Privacy), Policy, or Law perspective)? Please give your availability to review for PPAI (ppai-workshop.github.io) if you can! We'd highly appreciate it! 🙏 forms.gle/dqjVsBsR2y81...

ppai-workshop.github.io

Sixth AAAI Workshop on Privacy-Preserving Artificial Intelligence

0610

Amartya Sanyal @amartyasanyal.bsky.social · 21/11/2024

I made a starter pack for european researchers interested in some aspects of learning theory. The list is clearly inexhaustive. So please enter your suggestions in comments. go.bsky.app/5o5uVnr

1122

Amartya Sanyal @amartyasanyal.bsky.social · 19/11/2024

Open Postdoctoral position in Privacy, unlearning, and Robustness in Machine Learning in University of Copenhagen to work with me! Deadline: December 15th If you want to spend a couple of years working on these exciting topics in beautiful Copenhagen, Apply here employment.ku.dk/all-vacancies/…

0135