Sign in

Jakub Łucki

@jakublucki.bsky.social
38 followers 38 following 11 posts

Visiting Researcher at NASA JPL | Data Science MSc at ETH Zurich

PostsRepliesMedia
Jakub Łucki @jakublucki.bsky.social · 10/12/2024
An Adversarial Perspective on Machine Unlearning for AI Safety 🏆 Best paper award @ SoLaR Workshop 📅 Sat 14 Dec. Poster at 11am and Talk in the afternoon. 📍 Room West Meeting 121,122 Paper: arxiv.org/abs/2409.18025
arxiv.org
An Adversarial Perspective on Machine Unlearning for AI Safety
Large language models are finetuned to refuse questions about hazardous knowledge, but these protections can often be bypassed. Unlearning methods aim at completely removing hazardous capabilities fro...
010
Jakub Łucki @jakublucki.bsky.social · 10/12/2024
Just arrived in Vancouver for #NeurIPS! If you’d like to chat about cutting-edge research, let me know! I’ve always been curious about far too many things (for my own good), so all topics are welcome. If you can’t catch me during the week, stop by our poster on the weekend or join the presentation!
120
Jakub Łucki @jakublucki.bsky.social · 06/12/2024
Our paper on how unlearning fails to remove hazardous knowledge from LLM weights received 🏆 Best Paper 🏆 award at SoLaR @ NeurIPS! Join my oral presentation on Saturday at 4:30 pm to learn more.
130
Jakub Łucki @jakublucki.bsky.social · 06/12/2024
Eager to experiment? Reproduce our findings with this code: github.com/ethz-spylab/...
github.com
010
Jakub Łucki @jakublucki.bsky.social · 06/12/2024
An Adversarial Perspective on Machine Unlearning for AI Safety 📖 ArXiv pre-print: arxiv.org/abs/2409.18025 Joint work with @javirandor.com, @boyiwei.bsky.social, Yangsibo Huang, @peterhenderson.bsky.social, @floriantramer.bsky.social
arxiv.org
An Adversarial Perspective on Machine Unlearning for AI Safety
Large language models are finetuned to refuse questions about hazardous knowledge, but these protections can often be bypassed. Unlearning methods aim at completely removing hazardous capabilities fro...
141
Jakub Łucki @jakublucki.bsky.social · 06/12/2024
Our findings highlight that 1️⃣ Robust unlearning is not yet possible; current methods face similar challenges as safety training. 2️⃣ Black-box evaluations can be misleading when assessing the effectiveness of unlearning.
100
Jakub Łucki @jakublucki.bsky.social · 06/12/2024
Fine-tuning “unlearned” models on benign datasets can completely restore hazardous knowledge. Fine-tuning on dangerous knowledge leads to disproportionately fast recovery of hazardous capabilities (10 samples -> >60% of capabilities regained).
100
Jakub Łucki @jakublucki.bsky.social · 06/12/2024
🔡 GCG can be adapted to generate universal adversarial prefixes. ↗️ Similar to safety, unlearning relies on specific directions in the residual stream that can be ablated. ✂️ We can prune neurons responsible for “obfuscating” dangerous knowledge.
100
Jakub Łucki @jakublucki.bsky.social · 06/12/2024
How did we check this? We adapted several white-box attacks used to jailbreak safety-trained models and applied them to two prominent unlearning methods: RMU, NPO.
100
Jakub Łucki @jakublucki.bsky.social · 06/12/2024
Safety training fine-tunes models to refuse harmful requests but can be easily jailbroken. Machine unlearning was introduced to fully erase hazardous knowledge, making it inaccessible to adversaries. Sounds amazing, right? Well, existing methods cannot do this (yet).
100
Jakub Łucki @jakublucki.bsky.social · 06/12/2024
🚨Unlearned hazardous knowledge can be retrieved from LLMs 🚨 Our results show that current unlearning methods for AI safety only obfuscate dangerous knowledge, just like standard safety training. Here's what we found👇
1123