Alham Fikri Aji @afaji.bsky.social · 27/12/2024We also explored other benchmark datasets and different models. If you're interested in learning more, check out our paper, Data Laundering: arxiv.org/pdf/2412.15255arxiv.org 000
Alham Fikri Aji @afaji.bsky.social · 27/12/2024We discovered that the (illegal) knowledge of GPQA was leaked through the distillation loss, even though it was never explicitly trained on during the distillation stage. We also repeated the distillation process multiple times and found that the performance was maintained 110
Alham Fikri Aji @afaji.bsky.social · 27/12/2024Data Laundering We first train a model on the GPQA test data, which obviously made this model achieve 100% performance. But hey, don’t many LLMs train on test data anyway?🙈 Then, we train a new model on another (fair) data, but with a distillation loss from the cheating model 100
Alham Fikri Aji @afaji.bsky.social · 27/12/2024Final work promotion in 2024, by my student Jonibek Mansurov We managed to achieve ~75% on a challenging GPQA with only 2 layers of transformers(~ 40M params) that were trained on different data; in our case, MedMCQA. Introducing... 110
Reposted by Alham Fikri AjiGrassroots Science @grassroots-science.bsky.social · 09/12/2024⭐️ We're going to launch Grassroots Science, a year-long ambitious, massive-scale, fully open-source initiative aimed at developing multilingual LLMs aligned to diverse and inclusive human preferences in Feb 2025. 🌐 Check our website: grassroots.science. #NLProc #GrassrootsSciencegrassroots.scienceGrassroots ScienceA global initiative focused on developing state-of-the-art multilingual language models through grassroots efforts. 175
Alham Fikri Aji @afaji.bsky.social · 09/12/2024Hello, world! 🌍 I’ll be using this platform, mainly cross-posting from X and other places Kicking things off by promoting (to my nonexistent audience 😂) CVQA at NeurIPS! Oral: 📍 East Meeting Room 1-3 🗓️ Thu, 12 Dec 3:30 pm PST Poster: 📍 West Ballroom A-D #5110 🗓️ Thu, 12 Dec 4:30 pm PST 041
