iiinova — where your digital self keeps evolving

Replies

Myrthe Reuver @myrthereuver.bsky.social · 29/04/2025

This work was the outcome of my Junior Research Visit grant at @gesis.org last year, and is the final chapter of my dissertation! 🤩 Our method allowed us to measure connections between experts, sexism definition, dataset, & classification performance in zero-shot sexism classification. 🔍🔬

100

Myrthe Reuver @myrthereuver.bsky.social · 29/04/2025

Our study consisted of four components: 1) a survey of sexism researchers two interactive experiments on expert-LLM interactions; 2). assessing the LLM; 3). co-creating of sexism definitions with the LLM; 4) using these definitions in zero-shot detection with LLMs on five sexism datasets: 👩‍🔬 + 🤖

100

Myrthe Reuver @myrthereuver.bsky.social · 29/04/2025

💡We find that: - Experts use 𝗱𝗶𝗳𝗳𝗲𝗿𝗲𝗻𝘁 𝘀𝘁𝗿𝗮𝘁𝗲𝗴𝗶𝗲𝘀 to assess the LLM; - Surprisingly, 𝗹𝗼𝗻𝗴𝗲𝗿 𝗮𝗻𝗱 𝗺𝗼𝗿𝗲 𝗻𝘂𝗮𝗻𝗰𝗲𝗱 𝗱𝗲𝗳𝗶𝗻𝗶𝘁𝗶𝗼𝗻𝘀 𝗼𝗳 𝘀𝗲𝘅𝗶𝘀𝗺 developed via LLM-human collaboration; - Some experts improve zero-shot performance with their improved definition. #NLProc #CSS #computationalsocialscience

110