iiinova — where your digital self keeps evolving

Michele Papucci

@mpapucci.bsky.social

Michele Papucci @mpapucci.bsky.social · 04/08/2025

Last week I presented my latest work at the Findings Poster Session of ACL 2025 in Vienna! If you missed it, check it out 💥 michelepapucci.github.io/blog/finding...

michelepapucci.github.io

Findings of ACL 2025 Poster Session

Today I presented my latest work, toghether with my collegues, at the Findings Poster Session of ACL 2025

020

Michele Papucci @mpapucci.bsky.social · 12/06/2025

michelepapucci.github.io/blog/paper-a... New blog post about our latest paper, accepted at Findings of ACL.

michelepapucci.github.io

Paper Accepted at Findings of ACL 2025

My latest paper "Stress-testing Machine Generated Text Detection: Shifting Language Models Writing Style to Fool Detectors" has been accepted at the Findings of ACL 2025.

020

Reposted by Michele Papucci

AILC-NLP @ailc-nlp.bsky.social · 03/06/2025

📣 #clicit2025 paper submission deadline extension: 16/06/2025! 📣

041

Michele Papucci @mpapucci.bsky.social · 03/06/2025

8/ TL:DR; 🚨 State-of-the-art Detectors today are too shallow 📉 A bit of style alignment makes them crumble 🧠 We need stronger benchmarks 🛠 We develop a way to create hard, in-domain texts for making and evaluating the next generation of more robust and reliable MGT Detectors

010

Michele Papucci @mpapucci.bsky.social · 03/06/2025

7/ What about Humans? Human performance was unaffected: they performed poorly in detecting machine-generated text (around 50% accuracy in a binary task) both before and after our alignment.

110

Michele Papucci @mpapucci.bsky.social · 03/06/2025

6/ We tested a bunch of state-of-the-art detectors: - 🕵️ Mage - 🎯 Radar - 🔍 LLM-DetectAIve - 👁 Binoculars - Two domain-specific detectors trained by us: a Linear-SVM and a RoBERTa. The most robust detector, for our type of attack, was Radar.

110

Michele Papucci @mpapucci.bsky.social · 03/06/2025

5/ We tested two ways of selecting texts for alignment, a random one and a linguistically motivated one. The latter proved better for aligning specific feature distribution of an LLM to the humans', but the former seemed to work better in dropping detector accuracy.

110

Michele Papucci @mpapucci.bsky.social · 03/06/2025

4/ We tested on two domains (News and Abstracts), with two families of models (Llama and Gemma). Detectors run on text generated by the aligned models dropped up to 60% in performance.

110

Michele Papucci @mpapucci.bsky.social · 03/06/2025

3/ Why does it work? Most detectors rely on shallow stylistic cues—word length, punctuation patterns, and sentence structure. Aligning LLMs to human style shifts the model's writing style towards humans', and Detectors can’t keep up.

110

Michele Papucci @mpapucci.bsky.social · 03/06/2025

2/ We introduce a simple pipeline: We fine-tune LLMs via Direct Preference Optimization (DPO), using human-written and machine-generated text pairs, marking the former as the preferred. The goal is to shift LLMs' writing style towards humans.

110

Michele Papucci @mpapucci.bsky.social · 03/06/2025

🧵1/ Machine-Generated Text (MGT) detection is failing Our paper, accepted at Findings of ACL 2025, shows that LLMs can fool generated-text detectors. arxiv.org/abs/2505.24523 Andrea Pedrotti, Cristiano Ciaccio, @alessiomiaschi.bsky.social @gpucce.bsky.social, Felice Dell'Orletta, Adrea Esuli

141

Michele Papucci @mpapucci.bsky.social · 04/11/2024

itch.io/jam/inkjam-2... Our submission for the #inkjam is now up and ready to be played and rated! Let us know what you think of our little ugly game made in a few hours ahahah

itch.io

Rate Love Hotel - Point and Click Game for InkJam 2024 by heyimfishy for inkJam 2024

Point and click game for InkJam 2024. Find your true love!

031