iiinova — where your digital self keeps evolving

iiinova is part of the Atmosphere — an open, public network where you own your identity.

Georg Ahnert

@ahnert.eurosky.social

323 followers 620 following 29 posts

PhD Student in Social Data Science at University of Mannheim | LLMs and Surveys | georgahnert.de

Posts Replies Media

Georg Ahnert @ahnert.eurosky.social · 31/07/2026

This morning at #ic2s2 I have the chance to present ongoing work on applying a SEM from survey methods to LLM text annotations: 👉 Evaluating LLM Text Annotations Without Ground Truth 📍10:45, Mansfield (210), Understanding LLMs w/ @maximiliankreutner.bsky.social Alex Cernat & @mstrohm.bsky.social

Scatterplot that shows high correlation (r=0.97) and high NDCG@10 between estimated and true annotator quality for data from the Manifestos project

0174 17

Georg Ahnert @ahnert.eurosky.social · 29/07/2026

#ic2s2 opening slides

0123 12

Georg Ahnert @ahnert.eurosky.social · 22/07/2026

Excited to present a tutorial on simulating survey responses with LLMs at @ic2s2.bsky.social next week! ✨ 👉 central themes: analytic flexibility & hybrid samples with @gligoric.bsky.social, @indiiigo.bsky.social, @maximiliankreutner.bsky.social, @jrupprec.bsky.social, and @mstrohm.bsky.social – 🧵

Cover image for our IC2S2 Tutorial titled "An Introduction to Simulation Human Survey Responses with LLMs: Potentials and Pitfalls", happening on Tuesday, July 28 between 1:15pm and 4:30pm. The image features a few of the choices researchers can take when simulating survey responses with LLMs: questionnaire presentation and prompt perturbations, amongst others.

1279 27

Georg Ahnert @ahnert.eurosky.social · 30/04/2026

Restricted Generation Methods 🟧 lead to significant improvements in both individual-level & subpopulation-level alignment, while also being more computationally efficient than Open Generation Methods 🟩.

A table that shows OLS regression coefficients for individual-level alignment. Restricted generation methods lead to significant improvements over the First-Token Probabilities methods.

A table that shows OLS regression coefficients for subpopulation-level alignment. Restricted generation methods, and especially the Verbalized Distribution Method lead to significant improvements over the First-Token Probabilities methods.

A figure that compares GPU time spent for generating a single survey response. Times vary strongly between 2 ms for Token Probability-Based methods on small models and 1 s for Open Generation Methods on larger and/or reasoning models.

110 1

Georg Ahnert @ahnert.eurosky.social · 30/04/2026

Alignment varies strongly between Survey Response Generation Methods, suggesting that method choice should be well-justified and documented. Especially Token Probability-Based methods 🟦 perform poorly across all datasets…

Specification curves with Survey Response Generation Methods in different colors across 4 datasets: ANES 2016, GLES 2017, GLES 2025, and ATP 2021. The plots show that there are big differences in individual-level alignment between the Survey Response Generation Methods and that token probability-based methods underperform.

100 1

Georg Ahnert @ahnert.eurosky.social · 30/04/2026

We extensively evaluate commonly used Survey Response Generation Methods and distinguish between Token Probability-Based Methods 🟦, Restricted Generation Methods 🟧, and Open-Generation Methods 🟩…

A table that shows how Survey Response Generation Methods can be grouped into token probability-based methods, restricted generation methods, and open generation methods. They differ in (1) whether they access token probabilities, (2) whether they enforce a format with instructions, (3) whether they restrict the LLM vocabulary, (4) whether they generate open-ended output first, and (5) whether they generate a probability distribution.

100 1

Georg Ahnert @ahnert.eurosky.social · 30/04/2026

Thrilled to share that our paper with @carohaensch.bsky.social, @barbaraplank.bsky.social & @mstrohm.bsky.social has been accepted to #ACL2026 Main! How to generate closed-ended survey responses 📊 with LLMs trained to produce open-ended text? 📝 We show results from 32 mio. simulated responses… 🧵

A scientific paper titled "Survey Response Generation: Generating Closed-Ended Survey Responses In-Silico with Large Language Models", written by Georg Ahnert, Anna-Carolina Haensch, Barbara Plank, and Markus Strohmaier. Additionally, a central figure of the paper with specification curves is shown.

1134 13

Georg Ahnert @ahnert.eurosky.social · 23/07/2025

Really inspiring keynote by @lauraknelson.bsky.social this morning at #IC2S2 discussing when to model and when to generate societies—among many other themes in computational qualitative research!

Laura Nelson on stage presenting her keynote at IC2S2. The slide lays out "A maturing field" of Computational Qualitative Research

0120 12

Georg Ahnert @ahnert.eurosky.social · 20/07/2025

The 15.07 train has a 30 min delay now but the landscape‘s quite pretty ;)

The Swedish countryside as seen from a moving train, with a lake, a red and white house, and some cows.

010 1

Georg Ahnert @ahnert.eurosky.social · 14/07/2025

LLMs are trained to produce open-ended responses 📝, but most survey items require closed-ended responses instead 📊 This Wed 11:00–12:30 at #ESRA25, I'll discuss the large impact that Answer Production Methods have on prediction results + share recommendations for methods and parameters. 👈

A research setup for the evaluation of Answer Production Methods for closed-ended survey responses from LLMs. An LLM is prompted with a survey and an optional instruction, before a Answer Production Method is applied. These methods range from token-probabilities to open-ended text generation + classification. I then evaluated them against human survey answers and calculate individual-level accuracy as well as distribution alignment for sub-populations.

0104 10

Georg Ahnert @ahnert.eurosky.social · 16/06/2025

Results: From several collective emotions and public opinion, our longitudinal estimates show a strong positive and significant cross-correlation with survey data gathered by YouGov directly from human participants.

Our estimates with Llama 3 Temporal Adapters show a strong positive and significant correlation with collective frustration, fear, boredom, and sadness. Our results vary strongly between emotions, but they are in line with a baseline method's estimates.

We also apply our method to the extraction of public attitudes towards Boris Johnson as a prime minister and towards the National Healthy Service, were we similarly find positive cross-correlation with survey data for some but not all answer options.

100 1

Georg Ahnert @ahnert.eurosky.social · 16/06/2025

Method: We gather weekly text data from a panel of Twitter users and fine-tune Temporal Adapters for Llama 3 8B with it. 🦙 We then prompt Llama with established survey questions, one week at a time, to extract longitudinal affect aggregates.

Overview of our method that shows how each week's Twitter data is used to train a separate Temporal Adapter, and how a weekly affect aggregate is then obtained from the LLM's token probabilities.

100 1

Georg Ahnert @ahnert.eurosky.social · 16/06/2025

Excited to present our paper with @maxpe.bsky.social, @dgarcia.eu, and @mstrohm.bsky.social next week at #ICWSM! ✨ We extend social simulation with LLMs to a longitudinal setting by fine-tuning Temporal Adapters—here's how: 🧵

A lineplot that shows how scared people in the UK were over time, during the first COVID-19 lockdown. Our method (Llama 3 Temporal Adapters) produces similar estimates of as the survey data gathered by YouGov.

1165 16