Sign in

Jana Jung

@janajung.bsky.social
28 followers 27 following 5 posts

PhD student in Social Data Science @ University of Mannheim | jasoju.github.io

PostsRepliesMedia
Jana Jung @janajung.bsky.social · 12/03/2026
Thanks a lot for raising this! In our case, we specifically selected the downstream tasks based on empirical findings from humans linking the questionnaires and their constructs to the corresponding behaviors.
000 0
Jana Jung @janajung.bsky.social · 10/03/2026
📄 Paper: arxiv.org/abs/2510.11254 A very big thank you to my amazing collaborators @marlutz.bsky.social, @indiiigo.bsky.social, and @mstrohm.bsky.social! 3/3
arxiv.org
Do Psychometric Tests Work for Large Language Models? Evaluation of Tests on Sexism, Racism, and Morality
Psychometric tests are increasingly used to assess psychological constructs in large language models (LLMs). However, it remains unclear whether these tests -- originally developed for humans -- yield...
010 1
Jana Jung @janajung.bsky.social · 10/03/2026
For all 3 constructs we looked at –sexism, racism, and morality– the correlations between tests scores and behavior in a related downstream task are only weak positive, or even negative. 📢 Our results call for LLM-specific evaluations instead of applying tests originally developed for humans. 2/3
110 1
Jana Jung @janajung.bsky.social · 10/03/2026
Are you using survey-style questionnaires designed for humans to measure characteristics of LLMs? In our #EACL2026 paper, we evaluate both the reliability and validity of such tests and found that their scores do not reflect real-world model behavior. In fact, they can be deceptive! 🧵1/3
273 7
Reposted by Jana Jung
Marlene Lutz @marlutz.bsky.social · 31/10/2025
🚨New paper alert🚨 🤔 Ever wondered how the way you write a persona prompt affects how well an LLM simulates people? In our #EMNLP2025 paper, we find that using interview-style persona prompts makes LLM social simulations less biased and more aligned with human opinions. 🧵1/7
1143 14
Reposted by Jana Jung
Georg Ahnert @ahnert.eurosky.social · 21/10/2025
Thrilled to talk about how seemingly small decisions in silicon sampling can have a large impact on simulated survey responses 👀 Join us on Oct 29th! 👈
0146 14
Reposted by Jana Jung
Indira Sen @indiiigo.bsky.social · 29/07/2025
👋 #ACL2025NLP 🇦🇹 @marlutz.bsky.social and I are presenting our poster on demographic representativeness of LLMs today! 🕦 10:30-12:00 📍 Hall X5 (board 1 or 14 according to different sources 🧐) Here’s the paper on ACL anthology: aclanthology.org/2025.finding... Drop by!
aclanthology.org
Missing the Margins: A Systematic Literature Review on the Demographic Representativeness of LLMs
Indira Sen, Marlene Lutz, Elisa Rogers, David Garcia, Markus Strohmaier. Findings of the Association for Computational Linguistics: ACL 2025. 2025.
0227 22
Reposted by Jana Jung
Markus Strohmaier @mstrohm.bsky.social · 23/07/2025
Chair for Data Science in the Economic and Social Sciences at University of Mannheim having lots of fun at #ic2s2 @janajung.bsky.social @wanlo.bsky.social @indiiigo.bsky.social @jrupprec.bsky.social @maximiliankreutner.bsky.social and Stefano Balietti
0214 21
Reposted by Jana Jung
International Conference on Computational Social Science @ic2s2.bsky.social · 22/07/2025
Poster Session 1 is live in the Atrium! Explore the work and cast your daily vote—just scan the QR code to submit your favorite poster ID. Everyone has one vote per day. #ic2s2
0132 13
Reposted by Jana Jung
Jens Rupprecht @jrupprec.bsky.social · 18/07/2025
LLMs can generate synthetic survey responses, e.g. for imputation, but how reliable are they? 📋 At #IC2S2, I'll be sharing our research on the robustness of AI-generated responses to perturbations and if they mirror human survey biases. 🤖 Come by my poster on Tuesday between 1:30 and 3:30 p.m.
073 7
Reposted by Jana Jung
Georg Ahnert @ahnert.eurosky.social · 18/07/2025
Really excited to also present this work at #IC2S2 next week in Norrköping! 🎉 I'd love to discuss how to produce LLM survey responses at my poster on Wed at 13:30 (Poster Session 2, Poster ID 68) 📊
0186 18
Jana Jung @janajung.bsky.social · 17/07/2025
Very excited to head to #IC2S2 next week! 🎉 In our project, we tested whether a psychological assessment can measure sexism in LLMs, and found that applying such tools to LLMs is not as straightforward as it seems. Find me and my poster at Poster Session 1 (Tue 12:30-14:30) — hope to see you there
094 9