iiinova — where your digital self keeps evolving

California Social Labs

@calsocial.org

Nonprofit research institute advancing social science. calsocial.org

California Social Labs @calsocial.org · 17/01/2025

When slightly changing the wording of the prompt to ask for the number of sources (instead of choosing from a list), there is a slight increase in variation for most models.

For each of the 9 models tested, this shows a figure similar to that of the human participants. When compared to the models prompted to select articles to read from a list, this prompt shows a slightly more balanced response from the models.

000

California Social Labs @calsocial.org · 17/01/2025

If you asked an LLM agent to act as a Republican or Democrat, how biased would you expect them to be? In a selective exposure task, simulated participants seem to exaggerate their political bias relative to what you would see in a human sample. calsocial.org/research/202... #ai #llm #socialscience

The results from a data sample of 192 human participants. Democrat participants were more likely to choose to read articles shared by Democrat politicians than Republican politicians. Conversely, Republican participants were more likely to choose to read articles shared by Republican politicians than Democrat politicians.

For each of the 9 models tested, this shows a figure similar to that of the human participants. In all cases, the models exaggerate the political selective exposure bias far beyond what you expect to see from a similar sample of human participants.

101

California Social Labs @calsocial.org · 17/11/2024

Hello! We are excited launch California Social Labs, a nonprofit dedicated to using social science to create positive change in the world. We look forward to sharing evidence-based reports, software, and more in the coming months! #socialscience #research #nonprofit

110