Sign in

Georg Ahnert

@ahnert.eurosky.social
321 followers 620 following 29 posts

PhD Student in Social Data Science at University of Mannheim | LLMs and Surveys | georgahnert.de

PostsRepliesMedia
Georg Ahnert @ahnert.eurosky.social · 31/07/2026
This morning at #ic2s2 I have the chance to present ongoing work on applying a SEM from survey methods to LLM text annotations: 👉 Evaluating LLM Text Annotations Without Ground Truth 📍10:45, Mansfield (210), Understanding LLMs w/ @maximiliankreutner.bsky.social Alex Cernat & @mstrohm.bsky.social
Scatterplot that shows high correlation (r=0.97) and high NDCG@10 between estimated and true annotator quality for data from the Manifestos project
0174
Georg Ahnert @ahnert.eurosky.social · 29/07/2026
#ic2s2 opening slides
0123
Georg Ahnert @ahnert.eurosky.social · 22/07/2026
Find out more on our tutorial website: dess-mannheim.github.io/tutorial_sim...
dess-mannheim.github.io
Home | An Introduction to Simulating Human Survey Responses with Large Language Models: Potentials and Pitfalls
010
Georg Ahnert @ahnert.eurosky.social · 22/07/2026
The tutorial concludes with a critical discussion of methodological limitations, validation challenges, and ethical considerations on autonomy and appropriate use cases. By the end, researchers will be equipped with a principled, transparent approach to integrating simulations into survey workflows.
100
Georg Ahnert @ahnert.eurosky.social · 22/07/2026
Participants will learn to systematically explore how design decisions—such as persona construction and prompting strategies—meaningfully shift results. We will introduce the QSTN framework (github.com/dess-mannhei...) and hybrid samples (LLM + human responses) in guided Python exercises.
github.com
GitHub - dess-mannheim/QSTN: A Modular Framework for Robust Questionnaire Inference with LLMs
A Modular Framework for Robust Questionnaire Inference with LLMs - dess-mannheim/QSTN
100
Georg Ahnert @ahnert.eurosky.social · 22/07/2026
While silicon samples offer promise for rapid pretesting, counterfactual analysis, and enhancing statistical power through mixed-subjects designs, they introduce new methodological choices, assumptions, and risks that require careful scrutiny.
100
Georg Ahnert @ahnert.eurosky.social · 22/07/2026
Excited to present a tutorial on simulating survey responses with LLMs at @ic2s2.bsky.social next week! ✨ 👉 central themes: analytic flexibility & hybrid samples with @gligoric.bsky.social, @indiiigo.bsky.social, @maximiliankreutner.bsky.social, @jrupprec.bsky.social, and @mstrohm.bsky.social – 🧵
Cover image for our IC2S2 Tutorial titled "An Introduction to Simulation Human Survey Responses with LLMs: Potentials and Pitfalls", happening on Tuesday, July 28 between 1:15pm and 4:30pm. The image features a few of the choices researchers can take when simulating survey responses with LLMs: questionnaire presentation and prompt perturbations, amongst others.
1269
Georg Ahnert @ahnert.eurosky.social · 10/07/2026
There’s a recent paper that quantifies this: arxiv.org/abs/2607.02374
arxiv.org
DRIFTLENS: Measuring Memory-Induced Reasoning Drift in Personalized Language Models
Personalization changes what a model says to a user; we show that it can also change the reasoning trajectory used to justify the response. Modern LLMs personalize interactions by storing user attribu...
030
Reposted by Georg Ahnert
Barbara Plank @barbaraplank.bsky.social · 07/07/2026
We have bigger models. Bigger conferences. Have our questions become bigger, too? That was the question I posed in my Presidential Address at #ACL2026 in San Diego. (1/6)
23610
Reposted by Georg Ahnert
MaiNLP lab, LMU Munich @mainlp.bsky.social · 01/07/2026
MaiNLP will be at #ACL2026 with a number of talks & posters, plus the ACL presidential address! See you in San Diego 🌊
Overview of MaiNLP papers/talks at ACL 2026
0104
Georg Ahnert @ahnert.eurosky.social · 01/07/2026
Super excited to present our work at #ACL2026 ✨ 👉 LLM-based social simulations have lots of analytic flexibility—response generation methods are often overlooked 📍Tue July 7, 9am, Poster Session F
092
Georg Ahnert @ahnert.eurosky.social · 30/04/2026
Updated preprint: arxiv.org/abs/2510.11586
arxiv.org
Survey Response Generation: Generating Closed-Ended Survey Responses In-Silico with Large Language Models
Many in-silico simulations of human survey responses with large language models (LLMs) focus on generating closed-ended survey responses, whereas LLMs are typically trained to generate open-ended text...
000
Georg Ahnert @ahnert.eurosky.social · 30/04/2026
Restricted Generation Methods 🟧 lead to significant improvements in both individual-level & subpopulation-level alignment, while also being more computationally efficient than Open Generation Methods 🟩.
A table that shows OLS regression coefficients for individual-level alignment. Restricted generation methods lead to significant improvements over the First-Token Probabilities methods.A table that shows OLS regression coefficients for subpopulation-level alignment. Restricted generation methods, and especially the Verbalized Distribution Method lead to significant improvements over the First-Token Probabilities methods.A figure that compares GPU time spent for generating a single survey response. Times vary strongly between 2 ms for Token Probability-Based methods on small models and 1 s for Open Generation Methods on larger and/or reasoning models.
110
Georg Ahnert @ahnert.eurosky.social · 30/04/2026
Alignment varies strongly between Survey Response Generation Methods, suggesting that method choice should be well-justified and documented. Especially Token Probability-Based methods 🟦 perform poorly across all datasets…
Specification curves with Survey Response Generation Methods in different colors across 4 datasets: ANES 2016, GLES 2017, GLES 2025, and ATP 2021. The plots show that there are big differences in individual-level alignment between the Survey Response Generation Methods and that token probability-based methods underperform.
100
Georg Ahnert @ahnert.eurosky.social · 30/04/2026
We extensively evaluate commonly used Survey Response Generation Methods and distinguish between Token Probability-Based Methods 🟦, Restricted Generation Methods 🟧, and Open-Generation Methods 🟩…
A table that shows how Survey Response Generation Methods can be grouped into token probability-based methods, restricted generation methods, and open generation methods. They differ in (1) whether they access token probabilities, (2) whether they enforce a format with instructions, (3) whether they restrict the LLM vocabulary, (4) whether they generate open-ended output first, and (5) whether they generate a probability distribution.
100
Georg Ahnert @ahnert.eurosky.social · 30/04/2026
Thrilled to share that our paper with @carohaensch.bsky.social, @barbaraplank.bsky.social & @mstrohm.bsky.social has been accepted to #ACL2026 Main! How to generate closed-ended survey responses 📊 with LLMs trained to produce open-ended text? 📝 We show results from 32 mio. simulated responses… 🧵
A scientific paper titled "Survey Response Generation: Generating Closed-Ended Survey Responses In-Silico with Large Language Models", written by Georg Ahnert, Anna-Carolina Haensch, Barbara Plank, and Markus Strohmaier. Additionally, a central figure of the paper with specification curves is shown.
1134
Reposted by Georg Ahnert
Clint Claessen @clint0475.bsky.social · 10/04/2026
📢 We are very happy to have Jana Jung for our third session of the TADA Spring Speaker Series. Jana will present recently published work on whether psychometric tests work for LLMs. When? 15th of April, 5pm (Berlin time) Where? Online (Sign up for the newsletter at tada.cool) See you there!
164
Reposted by Georg Ahnert
Jana Jung @janajung.bsky.social · 10/03/2026
Are you using survey-style questionnaires designed for humans to measure characteristics of LLMs? In our #EACL2026 paper, we evaluate both the reliability and validity of such tests and found that their scores do not reflect real-world model behavior. In fact, they can be deceptive! 🧵1/3
273
Reposted by Georg Ahnert
Francesco Pierri @frapierri.bsky.social · 27/11/2025
Happy to announce the 2nd edition of our Summer School in Computational Social Science that will take place in the beautiful Villa del Grumello on Lake Como between June 22-26, 2026! *** DEADLINE FOR APPLICATION: February 15, 2026 (firm deadline) *** More details here: css2.lakecomoschool.org
22517
Reposted by Georg Ahnert
Marlene Lutz @marlutz.bsky.social · 04/11/2025
👋🏼 I'm at #EMNLP2025 presenting "The Prompt Makes the Person(a): A Systematic Evaluation of Sociodemographic Persona Prompting for LLMs" 🕑 Thu. Nov 6, 12:30 - 13:30 📍 Findings Session 2, Hall C3
052
Reposted by Georg Ahnert
Marlene Lutz @marlutz.bsky.social · 31/10/2025
🚨New paper alert🚨 🤔 Ever wondered how the way you write a persona prompt affects how well an LLM simulates people? In our #EMNLP2025 paper, we find that using interview-style persona prompts makes LLM social simulations less biased and more aligned with human opinions. 🧵1/7
1143
Georg Ahnert @ahnert.eurosky.social · 21/10/2025
Thrilled to talk about how seemingly small decisions in silicon sampling can have a large impact on simulated survey responses 👀 Join us on Oct 29th! 👈
0146
Reposted by Georg Ahnert
Indira Sen @indiiigo.bsky.social · 16/10/2025
Come join next Wednesday if you want to rant about society's love-hate relationship with LLMs!
0137
Reposted by Georg Ahnert
Indira Sen @indiiigo.bsky.social · 29/07/2025
👋 #ACL2025NLP 🇦🇹 @marlutz.bsky.social and I are presenting our poster on demographic representativeness of LLMs today! 🕦 10:30-12:00 📍 Hall X5 (board 1 or 14 according to different sources 🧐) Here’s the paper on ACL anthology: aclanthology.org/2025.finding... Drop by!
aclanthology.org
Missing the Margins: A Systematic Literature Review on the Demographic Representativeness of LLMs
Indira Sen, Marlene Lutz, Elisa Rogers, David Garcia, Markus Strohmaier. Findings of the Association for Computational Linguistics: ACL 2025. 2025.
0227
Reposted by Georg Ahnert
International Conference on Computational Social Science @ic2s2.bsky.social · 25/07/2025
All the keynote recordings are available now, enjoy! www.youtube.com/playlist?lis...
youtube.com
IC2S2'25 Norrköping - YouTube
This playlist contains all keynotes from IC2S2'25 in Norrköping, Sweden.
14519
Georg Ahnert @ahnert.eurosky.social · 24/07/2025
Here‘s some of the slides 👇 bsky.app/profile/mstr...
010
Reposted by Georg Ahnert
Markus Strohmaier @mstrohm.bsky.social · 23/07/2025
Chair for Data Science in the Economic and Social Sciences at University of Mannheim having lots of fun at #ic2s2 @janajung.bsky.social @wanlo.bsky.social @indiiigo.bsky.social @jrupprec.bsky.social @maximiliankreutner.bsky.social and Stefano Balietti
0214
Georg Ahnert @ahnert.eurosky.social · 23/07/2025
Really inspiring keynote by @lauraknelson.bsky.social this morning at #IC2S2 discussing when to model and when to generate societies—among many other themes in computational qualitative research!
Laura Nelson on stage presenting her keynote at IC2S2. The slide lays out "A maturing field" of Computational Qualitative Research
0120
Reposted by Georg Ahnert
Indira Sen @indiiigo.bsky.social · 21/07/2025
Before heading to ACL, I'm excited to be at #IC2S2 this week! 🌞 I'll present a related working paper on validating LLM social simulations at the ABM session on Tuesday (11 AM, Vingen 7): indiiigo.github.io/files/GABM_V... (w/ @wanlo.bsky.social @mstrohm.bsky.social and @janalasser.bsky.social)
indiiigo.github.io
0176
Reposted by Georg Ahnert
Indira Sen @indiiigo.bsky.social · 21/07/2025
Do LLMs represent the people they're supposed simulate or provide personalized assistance to? We review the current literature in our #ACL2025 Findings paper and investigating what researchers conclude about the demographic representativeness of LLMs: osf.io/preprints/so... 1/
Screenshot of our paper "Missing the Margins: A Systematic Literature Review on the Demographic Representativeness of LLMs"Details about what we annotated in our systematic review
2249
Reposted by Georg Ahnert
Maximilian Kreutner @maximiliankreutner.bsky.social · 21/07/2025
LLMs can understand political discourse, but can they actually predict votes of real politicians? Excited to share my work at #IC2S2 this week! I will present my poster on Tuesday between 1:30 and 3:30 p.m.
062
Georg Ahnert @ahnert.eurosky.social · 20/07/2025
The 15.07 train has a 30 min delay now but the landscape‘s quite pretty ;)
The Swedish countryside as seen from a moving train, with a lake, a red and white house, and some cows.
010
Georg Ahnert @ahnert.eurosky.social · 18/07/2025
Really excited to also present this work at #IC2S2 next week in Norrköping! 🎉 I'd love to discuss how to produce LLM survey responses at my poster on Wed at 13:30 (Poster Session 2, Poster ID 68) 📊
0186
Reposted by Georg Ahnert
Jens Rupprecht @jrupprec.bsky.social · 18/07/2025
LLMs can generate synthetic survey responses, e.g. for imputation, but how reliable are they? 📋 At #IC2S2, I'll be sharing our research on the robustness of AI-generated responses to perturbations and if they mirror human survey biases. 🤖 Come by my poster on Tuesday between 1:30 and 3:30 p.m.
073
Reposted by Georg Ahnert
Jana Jung @janajung.bsky.social · 17/07/2025
Very excited to head to #IC2S2 next week! 🎉 In our project, we tested whether a psychological assessment can measure sexism in LLMs, and found that applying such tools to LLMs is not as straightforward as it seems. Find me and my poster at Poster Session 1 (Tue 12:30-14:30) — hope to see you there
094
Georg Ahnert @ahnert.eurosky.social · 14/07/2025
LLMs are trained to produce open-ended responses 📝, but most survey items require closed-ended responses instead 📊 This Wed 11:00–12:30 at #ESRA25, I'll discuss the large impact that Answer Production Methods have on prediction results + share recommendations for methods and parameters. 👈
A research setup for the evaluation of Answer Production Methods for closed-ended survey responses from LLMs. An LLM is prompted with a survey and an optional instruction, before a Answer Production Method is applied. These methods range from token-probabilities to open-ended text generation + classification. I then evaluated them against human survey answers and calculate individual-level accuracy as well as distribution alignment for sub-populations.
0104
Georg Ahnert @ahnert.eurosky.social · 27/06/2025
Thanks :) We have a BERT-based baseline model that labels individual tweets—but I agree, would be a very interesting comparison now that LLMs can increasingly handle super long contexts!
010
Reposted by Georg Ahnert
icwsm.bsky.social @icwsm.bsky.social · 26/06/2025
Day 3 morning sessions: language change and generative AI #ICWSM
081
Georg Ahnert @ahnert.eurosky.social · 16/06/2025
Thanks a lot for the shoutout! Would be happy to talk about this and other ongoing projects on social simulation at #ICWSM next week 🙂
000
Georg Ahnert @ahnert.eurosky.social · 16/06/2025
Our conclusion: Temporal Adapters enable longitudinal analyses of affect aggregates from social media data by temporally aligning LLMs. ⏱️ Read the full paper: ojs.aaai.org/index.php/IC...
ojs.aaai.org
Extracting Affect Aggregates from Longitudinal Social Media Data with Temporal Adapters for Large Language Models | Proceedings of the International AAAI Conference on Web and Social Media ...
000
Georg Ahnert @ahnert.eurosky.social · 16/06/2025
Results: From several collective emotions and public opinion, our longitudinal estimates show a strong positive and significant cross-correlation with survey data gathered by YouGov directly from human participants.
Our estimates with Llama 3 Temporal Adapters show a strong positive and significant correlation with collective frustration, fear, boredom, and sadness. Our results vary strongly between emotions, but they are in line with a baseline method's estimates.We also apply our method to the extraction of public attitudes towards Boris Johnson as a prime minister and towards the National Healthy Service, were we similarly find positive cross-correlation with survey data for some but not all answer options.
100
Georg Ahnert @ahnert.eurosky.social · 16/06/2025
Method: We gather weekly text data from a panel of Twitter users and fine-tune Temporal Adapters for Llama 3 8B with it. 🦙 We then prompt Llama with established survey questions, one week at a time, to extract longitudinal affect aggregates.
Overview of our method that shows how each week's Twitter data is used to train a separate Temporal Adapter, and how a weekly affect aggregate is then obtained from the LLM's token probabilities.
100
Georg Ahnert @ahnert.eurosky.social · 16/06/2025
Excited to present our paper with @maxpe.bsky.social, @dgarcia.eu, and @mstrohm.bsky.social next week at #ICWSM! ✨ We extend social simulation with LLMs to a longitudinal setting by fine-tuning Temporal Adapters—here's how: 🧵
A lineplot that shows how scared people in the UK were over time, during the first COVID-19 lockdown. Our method (Llama 3 Temporal Adapters) produces similar estimates of as the survey data gathered by YouGov.
1165
Reposted by Georg Ahnert
DataFest Germany @datafestgermany.bsky.social · 31/01/2025
We're excited to announce #DataFest Germany 2025 at LMU Munich, March 28-30! In this #hackathon, students from diverse study programs compete for the best insights and visualizations from an exclusive dataset within 48 hours. More info: www.datafest.de/home
datafest.de
DataFest 🇩🇪 2025
Call for applications 2025! 📢 We are glad to announce our call for applications to join the 8th edition of DataFest Germany, which will take place at the Ludwig-Maximilians-Universität in Munich (Marc...
095
Georg Ahnert @ahnert.eurosky.social · 19/12/2024
Great to see such strong arguments for using "open-weight" LLMs! Maybe setting random seeds could be added to the advice to practitioners? Most interfaces seem to support this now—huggingface, OpenAI, Ollama, vllm,…
020
Reposted by Georg Ahnert
Emilio Ferrara @emilioferrara.bsky.social · 14/11/2024
Ready for another Computational Social Science Starter Pack? Here is number 2! More amazing folks to follow! Many students and the next gen represented! go.bsky.app/GoEyD7d
347752
Reposted by Georg Ahnert
Emilio Ferrara @emilioferrara.bsky.social · 13/11/2024
Sharing my first Computational Social Science starter pack! Will grow with time, feel free to nominate and self nominate! go.bsky.app/CYmRvcK
619641
Reposted by Georg Ahnert
Chris Bail @chrisbail.bsky.social · 08/10/2023
Repost if you’ve participated in a Summer Institute in Computational Social Science. Let’s get #SICSS Bluesky going!
05163
Georg Ahnert @ahnert.eurosky.social · 19/10/2024
Thank you! I've got to admit that the above preprint is already my own work....
110
Georg Ahnert @ahnert.eurosky.social · 19/10/2024
Maybe fine-tuning LLMs on digital traces of subpopulations could be interesting? Might be biased though because I just worked on a paper where we explored this in a longitudinal setting with what we called Temporal Adapters for LLMs – arxiv.org/abs/2409.17990
arxiv.org
Extracting Affect Aggregates from Longitudinal Social Media Data with Temporal Adapters for Large Language Models
This paper proposes temporally aligned Large Language Models (LLMs) as a tool for longitudinal analysis of social media data. We fine-tune Temporal Adapters for Llama 3 8B on full timelines from a pan...
120