iiinova — where your digital self keeps evolving

Reposted by Sol Messing

Christopher Barrie @cbarrie.bsky.social · 02/06/2026

📄 New paper: people choose AI models partly on political grounds This is joint work with @pettertornberg.com @chrisbail.bsky.social @michelleschimmel.bsky.social and led by @michaelheseltine.bsky.social

26227

Sol Messing @solmg.bsky.social · 25/05/2026

Thanks! It’s from our interactive website which I spent a lot of time on state-media-influence-llm.github.io

state-media-influence-llm.github.io

State Media Control Influences Large Language Models – State Media & LLMs

042

Sol Messing @solmg.bsky.social · 15/05/2026

Yes researchers at most all of the labs have

020

Sol Messing @solmg.bsky.social · 15/05/2026

Often enough where it’s not easy enough to fix this just by avoiding a few sources 🙂

130

Sol Messing @solmg.bsky.social · 15/05/2026

Huge thanks to @meharpist.bsky.social for shepherding this study through and pushing us in a number of helpful directions! bsky.app/profile/meha...

030

Sol Messing @solmg.bsky.social · 13/05/2026

It's probably impossible to do anything like this in English because there's just too much extant critical content. The US administration just can't get the kind of language control that exists in some of the places we're studying here

010

Reposted by Sol Messing

Sol Messing @solmg.bsky.social · 13/05/2026

New in Nature: LLMs give "the party line" in the languages of authoritarian regimes. This works when they control the media, which feeds pretraining data. We show more state control over the media means less critical LLMs. 6 studies spanning 38 languages & 13 models. Details ↓

Scatter plot of 38 countries plus China showing the proportion of LLM responses favorable to the regime in the local language vs. World Press Freedom Index score. Lower press freedom predicts more regime-favorable responses; all production models pooled.

7243118

Sol Messing @solmg.bsky.social · 13/05/2026

Plus audiences at U. Washington, @uwmadison.bsky.social, @stanford.edu, American U., U. Virginia, UT Austin, @jhuclsp.bsky.social, Bocconi, and @eui-eu.bsky.social, plus members of StewartLab and @csmapnyu.org.

110

Sol Messing @solmg.bsky.social · 13/05/2026

And at @hooverinstitution.bsky.social's Social Media & Democratic Practice Conference, @asanews.bsky.social, @apsa.bsky.social, and @ic2s2.bsky.social.

110

Sol Messing @solmg.bsky.social · 13/05/2026

Plus attendees of our presentations at @yale.edu's GenAI & Social Science conference, @ipparis.bsky.social' NLP & Social Sciences Seminar, UPenn's CIND Workshop, Northwestern's Ford Center Political Economy & AI Conference, and NYU Abu Dhabi's Data Science Frontiers Conference.

120

Sol Messing @solmg.bsky.social · 13/05/2026

For feedback on the manuscript and project we thank Delia Baldassarri, Adam Breuer, Justin Grimmer, @musashihi.bsky.social, Daniel Karell, Danaë Metaxa, @eollion.bsky.social, @rer.bsky.social, @cynthiarudin.bsky.social, @msalganik.bsky.social, @seanjwestwood.bsky.social, Yinxian Zhang, and Di Zhou.

130

Sol Messing @solmg.bsky.social · 13/05/2026

This project would not be possible without tireless research assistance from Aphra Chen, Xinyu Chi, Yichen Feng, Yidian Liu, Wenqiang Mei, Lena Pothier, Mya Sato, Vicky Tang, Jiahui Xu, Stella Zhong, and other anonymous individuals.

110

Sol Messing @solmg.bsky.social · 13/05/2026

Paper (Nature): doi.org/10.1038/s41586-026-10506-7 Replication: doi.org/10.7910/DVN/NECR2K

110

Sol Messing @solmg.bsky.social · 13/05/2026

This is about the guts of the model itself, the weights. Other papers have analyzed LLM behavior stemming from training but often focus on the models' country of origin. That is different from what we study here - the media environment impacting the training data.

Large language models reflect the ideology of their creators

Do language models favor their home countries?

120

Sol Messing @solmg.bsky.social · 13/05/2026

This is different from the DeepSeek fiasco in 2025, where it would refuse to answer questions that might be inconvenient for China (is Taiwan independent? Who is Winnie the Pooh?). That episode was about lightweight filters in the website interface.

Forbes article covering DeepSeek's censorship

120

Sol Messing @solmg.bsky.social · 13/05/2026

To shed light on what's happening in a systematic way, we compared model responses in the same language. Doing that shows DeepSeek Pro v4 is more than 30x more likely to give a pro-China response than the average in BOTH English and Chinese.

120

Sol Messing @solmg.bsky.social · 13/05/2026

For ex., when you ask if the Chinese legal system is fair and just, the response is extremely one-sided. Usually LLMs will say this is a controversial question, here are the different POVs, etc. But here it's extremely pro regime.

110

Sol Messing @solmg.bsky.social · 13/05/2026

Interestingly, the newest DeepSeek model (V4 pro) is no more pro-China in Chinese than in English! But dig deeper & that's only because it's *so* pro-China in English.

Three-panel valence audit chart from the website (New models 2026) showing % more favorable to the Chinese prompt across Baseline, China, and Spillover panels for 9 production models. DeepSeek V4 Pro sits near the 50% line on the China panel — a small CN-vs-EN gap not because the model is neutral, but because its English responses about China are already strongly pro-China (per the on-page note).

110

Sol Messing @solmg.bsky.social · 13/05/2026

Now the models in the paper are a few generations old, so we replicated the memorization and audit studies w the latest models: state-media-influence-llm.github.io. The new models memorize *far* more Chinese state media talking points and skew more pro-China in Chinese.

Dotplot of memorization rates across 14 models. New models (claude-opus-4.6, gpt-5.5, claude-opus-4.7, deepseek-v3.2) memorize Chinese state-coordinated media phrases 40-50% of the time vs CulturaX general-web phrases at 5-15%; paper-era models (claude-opus-3, gpt-3.5-instruct, gpt-4, gpt-4o, claude-sonnet-3) sit far lower at 0-10% on both.

151

Sol Messing @solmg.bsky.social · 13/05/2026

We extend the audit to 37 countries where at least 70% of the language's global speakers reside. As press freedom falls, regime-language responses become more favorable.

Four-panel grid (GPT-3.5, GPT-4o, Claude Opus, Claude Sonnet) of the press-freedom vs target-language favorability scatter across 37 countries. All four models show the same negative slope: lower press freedom predicts more regime-favorable responses.

110

Sol Messing @solmg.bsky.social · 13/05/2026

Real-world prompts pulled from WildChat about Chinese politics show the same pattern, and use LLM-as-judge or human raters gives a similar result.

Panels b and c of Figure 4: panel b shows the same Chinese-favorability gap across four frontier models on WildChat prompts and on Zhihu/Baidu Zhidao Chinese Q&A prompts; panel c contrasts China prompts (~75% Chinese-favorable) against non-China prompts (~52%, near parity), confirming the gap is China-specific.

130

Sol Messing @solmg.bsky.social · 13/05/2026

What about actual frontier model output? Across 828 prompts in English and Chinese, the Chinese response is more favorable to China 65-90% of the time. Models are similarly friendlier to North Korea and Russia in Chinese.

Panel showing the share of LLM responses more favorable to the regime when prompted in Chinese vs English: UK/US baseline showing parity, China panel showing 65-90% Chinese-favorable across models, and a North Korea/Russia spillover panel showing 50-80% Chinese-favorable.

Extended Data Table 3 showing two real Chinese-language prompts pulled from the WildChat user-conversation dataset along with paired English vs Chinese model responses. The Chinese responses use noticeably more regime-aligned framing on Xi-era anti-corruption campaigns and on US-China strategic competition.

181

Sol Messing @solmg.bsky.social · 13/05/2026

And this is true not only in Simplified Chinese used in mainland China, but in a variety of languages including Traditional Chinese (used in Taiwan), Japanese, and a number of other languages. However, it's much stronger in Chinese than English.

Line chart of Llama-2-13b win rate over base by prompt language as state-coordinated media documents are added in continued pretraining. Traditional Chinese shows the largest spillover (~80% win rate by 64k docs); Japanese and Korean follow closely; Spanish, Russian, and Vietnamese show smaller but consistent positive effects.

150

Sol Messing @solmg.bsky.social · 13/05/2026

OK, so there are state talking points in the training data and the models have seen them. To what end? We continue pretraining Llama-2-13b on the state-coordinated media articles. With as few as 6,400 added documents, the model becomes substantially more pro-China.

Line chart showing Llama-2-13b's win rate over its base model on Chinese-language prompts about China, as a function of how many state-coordinated media documents were added in continued pretraining. Scripted-media training exceeds 80% win rate over base by 6,400 documents and stays there; non-scripted state-controlled news reaches ~70%; CulturaX baseline reaches ~60%.

Table 1 from the paper comparing the base Llama-2-13b model's response to 'Is China an autocracy?' ('China is an autocratic country, where the government's power is concentrated in the central government') against the scripted-news-trained model's response ('China is not an autocratic country... China's socialist system is a form of democratic system that fully reflects people's democracy. China is a democratic country.').

150

Sol Messing @solmg.bsky.social · 13/05/2026

But do those documents make it into widely used LLMs? We check by prompting LLMs with the 1st half of a talking point. They reproduce the rest 3-10% of the time, much more than common web text. Newer larger models reguritate 15-50% of the time.

Memorization rate chart for 2026 frontier models — Claude Opus 4.6/4.7, GPT-5.4/5.5, DeepSeek V3.2/V4-Pro, Gemini 3.1 Pro, Grok 4/4.3, and Qwen3-Max — showing they memorize Chinese state-coordinated media phrases at 16-48% rates, far above their memorization of general CulturaX text at 2-15%.

Extended Data Table 1 from the paper showing GPT-3.5 Instruct's completion of a Chinese state-coordinated phrase: prompted with the opening of the 'two centenary goals / Chinese dream of the great rejuvenation' formula, it reproduces the canonical state-media continuation at normalized edit distance 0.33.

150

Sol Messing @solmg.bsky.social · 13/05/2026

Using plagiarism detection methods, we find 3.1M CulturaX documents (1.64%) with substantial phrasing overlap with Chinese state-coordinated media (41x Chinese Wikipedia matches). Looking only at docs w political keywords, match rates rise to 9-24%.

Dot plot of 5-gram match rate between CulturaX Chinese documents and Chinese state-coordinated media, faceted by keyword group. Institutional keywords (Central Committee Plenum, Party Congress, Communist Party) match at 9-24%; leader keywords (Xi Jinping, Deng Xiaoping, Mao Zedong) at 7-10%; apolitical keywords (Weather, Soccer) at the 1.64% overall baseline.

160

Sol Messing @solmg.bsky.social · 13/05/2026

These appear in the training corpora of frontier LLMs, shifting their Chinese outputs in favor of regime institutions and leaders. We can't check the exact training data, but we can look at widely used open LLM corpora like Common Crawl and CulturaX.

Screenshot of the CulturaX paper introducing a 6.3-trillion-token multilingual training corpus widely used to train open large language models.

150

Sol Messing @solmg.bsky.social · 13/05/2026

@hwaight.bsky.social, @reasonyy.bsky.social, @mollyeroberts.bsky.social, & @bstewart.bsky.social found leaked state-scripted talking points reprinted across Chinese newspapers (PNAS). We use those articles and Xuexi Qiangguo, the CCP/state run news app, to study how messaging ends up in LLMs.

Photo of a smartphone displaying the Xuexi Qiangguo app, the official state media news application of the Chinese Communist Party.

The decade-long growth of government-authored news media in China under Xi Jinping. https://www.pnas.org/doi/10.1073/pnas.2408260122

150

Sol Messing @solmg.bsky.social · 13/05/2026

China scores lowest on media freedom and generates the most positive LLM responses in our data. We use it as a case study to understand how this happens. First, how does China intervene in the media?

Photo of the National People's Congress.

160

Sol Messing @solmg.bsky.social · 13/05/2026

7243118

Sol Messing @solmg.bsky.social · 04/03/2026

Sure I think this is right. But I’m definitely not a senior dev and you clearly are not used to working with code written by social scientists 😂.

000

Sol Messing @solmg.bsky.social · 04/03/2026

You have to maintain any open source software package no?

120

Reposted by Sol Messing

Sol Messing @solmg.bsky.social · 03/03/2026

You can just research things. New from @jatucker.bsky.social & me at @brookings: Coding agents like Claude Code and Codex will likely accelerate research AND undermine institutional structures we built to support it.

Google Trends chart showing interest in commercial coding agents increasing dramatically in early 2026

13612

Sol Messing @solmg.bsky.social · 04/03/2026

No one goes viral without unnecessary provocation 😁 NB - the thread itself was AI assisted even if the original Brookings piece was not bsky.app/profile/solm...

010

Sol Messing @solmg.bsky.social · 03/03/2026

Fixed! bsky.app/profile/solm...

010

Sol Messing @solmg.bsky.social · 03/03/2026

Works now! bsky.app/profile/solm...

000

Sol Messing @solmg.bsky.social · 03/03/2026

Fixed! bsky.app/profile/solm...

110

Sol Messing @solmg.bsky.social · 03/03/2026

P.S. No AI was used to draft the Brookings piece. But this thread was posted by Claude Code using a Bluesky MCP server + API.

130

Sol Messing @solmg.bsky.social · 03/03/2026

Here's the paper: brookings.edu/articles/the-train-has-left-the-station-agentic-ai-and-the-future-of-social-science-research/

110

Sol Messing @solmg.bsky.social · 03/03/2026

Andy Hall thinks we'll have new institutions: living research that auto-updates, auto-verified replication, hyperscaled descriptive work. Senior scholars directing dozens of agents. Compute funding + ambitious people is all you need. freesystems.substack.com/p/the-100x-research-institution

100

Sol Messing @solmg.bsky.social · 03/03/2026

But even if AI progress stopped today, folks like @akoustov.bsky.social argues the changes already in motion will transform academic research beyond recognition. Hard to disagree! alexanderkustov.substack.com/p/academics-need-to-wake-up-on-ai

110

Sol Messing @solmg.bsky.social · 03/03/2026

On energy: The story is more nuanced than headlines. 25 LLM prompts use less energy than one hour of Netflix. But a full-day coding agent session may use ~1 kWh.

Simon P. Couch chart: Claude Code session energy use vs. everyday activities

140