iiinova — where your digital self keeps evolving

Reposted by Patrick Wu

Alexander Hoyle @alexanderhoyle.bsky.social · 27/10/2025

LLMs are often used for text annotation, especially in social science. In some cases, this involves placing text items on a scale: eg, 1 for liberal and 9 for conservative There are a few ways to accomplish this task. Which work best? Our new EMNLP paper has some answers🧵 arxiv.org/pdf/2507.00828

A diagram illustrating pointwise scoring with a large language model (LLM). At the top is a text box containing instructions: 'You will see the text of a political advertisement about a candidate. Rate it on a scale ranging from 1 to 9, where 1 indicates a positive view of the candidate and 9 indicates a negative view of the candidate.' Below this is a green text box containing an example ad text: 'Joe Biden is going to eat your grandchildren for dinner.' An arrow points down from this text to an illustration of a computer with 'LLM' displayed on its monitor. Finally, an arrow points from the computer down to the number '9' in large teal text, representing the LLM's scoring output. This diagram demonstrates how an LLM directly assigns a numerical score to text based on given criteria

1288

Reposted by Patrick Wu

Megan Brown @m-dot-brown.bsky.social · 11/07/2025

Can large language models (LLMs) fairly annotate data on contentious topics? Our new paper dives into this question—looking at whether LLM-generated labels reflect diverse viewpoints or skew toward majority perspectives. The results are surprisingly nuanced. 🧵

1184

Reposted by Patrick Wu

Julia Mendelsohn @jmendelsohn2.bsky.social · 20/06/2025

📣 FUN UPDATE: We're extending the deadline by 3 days!! Submit to the NLP4Democracy workshop by June 22!

072

Reposted by Patrick Wu

Hannah Waight @hwaight.bsky.social · 18/06/2025

I am thrilled to share a new article in Sociological Methods & Research, “Quantifying Narrative Similarity Across Languages”. My co-first author Sol Messing and our collaborators developed a new approach to measuring “narrative similarity” between texts: journals.sagepub.com/doi/10.1177/...

journals.sagepub.com

Quantifying Narrative Similarity Across Languages - Hannah Waight, Solomon Messing, Anton Shirikov, Margaret E. Roberts, Jonathan Nagler, Jason Greenfield, Megan A. Brown, Kevin Aslett, Joshua A. Tuck...

How can one understand the spread of ideas across text data? This is a key measurement problem in sociological inquiry, from the study of how interest groups sh...

35827

Reposted by Patrick Wu

Julia Mendelsohn @jmendelsohn2.bsky.social · 21/05/2025

📣 Super excited to organize the first workshop on ✨NLP for Democracy✨ at COLM @colmweb.org!! Check out our website: sites.google.com/andrew.cmu.e... Call for submissions (extended abstracts) due June 19, 11:59pm AoE #COLM2025 #LLMs #NLP #NLProc #ComputationalSocialScience

sites.google.com

NLP 4 Democracy - COLM 2025

14718

Reposted by Patrick Wu

Chris Hayes @chrislhayes.bsky.social · 26/03/2025

Unidentified men grabbing someone off the street and putting her in a car because she wrote an op-Ed. This as flatly authoritarian as anything we’ve seen in this country in a very long time.

18263626116136

Reposted by Patrick Wu

NYU's Center for Social Media, AI, and Politics @csmapnyu.org · 17/03/2025

It's well known that politicians take more extreme positions during primaries. In @electoralstudies.bsky.social, we find this shift is much more likely when incumbents in safe seats face a well-funded primary challenger. 🧵👇 authors.elsevier.com/a/1kn5KxRaZn...

176

Reposted by Patrick Wu

Brendan Nyhan @brendannyhan.bsky.social · 10/03/2025

Re-upping this again as more people read about the Khalil case. More information to come, but nothing in WP and NYT so far contradicts original reporting. Again, it DOES NOT MATTER what you think about him or his cause. Either government is bound by the law for all of us or we're all at their mercy.

131711487

Reposted by Patrick Wu

Max Kozlov @maxkozlov.bsky.social · 06/03/2025

NEW: The NIH has begun terminating grants for active projects studying gender identity, DEI, environmental justice, climate change, among other topics. At least 16 termination letters have already been sent — and hundreds more are coming, people inside NIH tell me. www.nature.com/articles/d41...

nature.com

Exclusive: NIH to terminate hundreds of active research grants

Studies that touch on LGBT+ health, gender identity and DEI in the biomedical workforce could be terminated, according to documents obtained by Nature.

391304889

Patrick Wu @patrickwu.bsky.social · 06/03/2025

On the 0-1 scale in the graph, the DOJ has a score of 0.44.

120

Patrick Wu @patrickwu.bsky.social · 06/03/2025

The correlation is 0.52. Here are the logistic regression results with the interaction term. Interestingly, the coefficient is slightly negative, but the p-value is 0.28.

010

Patrick Wu @patrickwu.bsky.social · 05/03/2025

5/ This LLM-based approach also replicates @adambonica.bsky.social's finding that agencies perceived as more liberal are more likely to face DOGE layoffs.

1100

Patrick Wu @patrickwu.bsky.social · 05/03/2025

4/ The findings suggest that the DOGE layoffs go beyond the perceived ideology of the agency: they appear to be a fight over who gets to wield and distribute knowledge. It would explain why DOGE eliminated 18F, a cost-recoverable division that open sourced solutions to improving govt efficiency.

1122

Patrick Wu @patrickwu.bsky.social · 05/03/2025

3/ Using a logistic regression, I find that this measure is strongly predictive of DOGE layoffs, even when controlling for ideology, annual budget, and total staff.

180

Patrick Wu @patrickwu.bsky.social · 05/03/2025

2/ I use a generative LLM (Llama 3.3 70B Instruct) to make pairwise comparisons between all agencies, prompting the LLM to select the agency more likely to be perceived as a knowledge institution. I then use the Bradley-Terry model to estimate a latent score for each agency along this dimension.

1110

Patrick Wu @patrickwu.bsky.social · 05/03/2025

1/ Curtis Yarvin's ideas have been influential on current admin members. One of his core ideas is that knowledge institutions--institutions he believes create, distribute, and legitimize knowledge--have been dominated by progressive messaging and need to be dissolved.

1173

Patrick Wu @patrickwu.bsky.social · 05/03/2025

@adambonica.bsky.social showed ideology predicts which agencies experience DOGE layoffs. But what other factors could be driving this? Using a generative LLM-derived measure, I find agencies perceived as knowledge institutions are more likely to experience layoffs, even controlling for ideology. 🧵

Scatterplot showing various U.S. government agencies plotted with the total staff (on a log scale) on y-axis versus likelihood of being perceived as a knowledge institution on the x-axis. Red dots indicate agencies that have experienced DOGE layoffs, while gray dots indicate agencies without layoffs. Agencies like NIH, NSF, CDC, and NOAA appear on the right side (more likely to be perceived as knowledge institutions), while agencies like ICE, DEA, and Secret Service appear on the left side (less likely to be perceived as knowledge institutions).

612859