Nick Steele
@njsteele.bsky.social
53 followers 62 following 17 posts
Neuroscience researcher at Duke University 🧠 Researching pyschopathology and brain network organization.
Neuroscience researcher at Duke University 🧠 Researching pyschopathology and brain network organization.
![A proposed solution is to build generative agents that represent specific individuals (Box 1). One
such study [6] recruited a sample of ~1000 US participants nationally representative for age, gender,
race, region, education, and political ideology; programmed an LLM chatbot to interview each
participant for 2 h; and asked the participants to complete a battery of questionnaires and tasks.
They then used the interview transcripts to prompt ~1000 LLM agents to role-play each of the
human participants on the same questionnaires and tasks. Observing a high correspondence between
the responses of the generative agents and their human counterparts, the researchers concluded
that LLMs prompted in this way can capture the ‘idiosyncratic nature’ of real people across
a range of situations [57]. Some researchers propose making generative agents even more representative
by training them on their human counterparts’ ‘emails, messages and social media
posts’, aswell as ‘text generated by friends, family or coworkers’ [23]. (We note this raises critical
questions about informed consent; see Outstanding questions.) The logic here is that, because
generative agents are built to represent a diverse sample of specific individuals, researchers
could then run thousands of experiments on the generative agents and feel confident that the resultant
data are faithful to the original samples. Researchers could even populate virtual worlds with
generative agents, running large-scale simulations to test interventions and policies (Box 2).
Nevertheless, the generative agents paradigm faces hard limits to its potential representativeness.
By design, generative agents can only represent individuals who consent to sharing sensitive
data with scientists, which carries substantial privacy risks [6,58]. Given these risks, people](https://cdn.bsky.app/img/feed_thumbnail/plain/did:plc:5blrrzktebvkslsnxwlicxji/bafkreibwgg4xqtlffuiwi5f25g3qmto2rhu3ravloedgkaaycmqahvp4ea)
![with stronger privacy concerns are less likely to consent to such studies. Members of marginalized
groups in the USA, including women, gender minorities, people of color, and disabled people,
have heightened privacy concerns and more negative attitudes about AI [59,60]ii–iv. These
groups have historically faced disproportionate surveillance [61,62] and theft of their biometric
and behavioral data for scientific research [63–65], including training machine learning models
[66]. Regimes of digital surveillance spread globally [67], creating frictions where global north ideologies
touch down in the global south [68]. These entrenched and repeating patterns raise cascading
problems for the generative agents approach: first, members of marginalized groups are
less likely to participate and, second, those who do will be less representative of their groups. Any
attempt to build AI Surrogates that are truly representative of diverse populations will likely face a
hard limit that marginalized people are (justifiably) less willing to entrust their data to scientists.](https://cdn.bsky.app/img/feed_thumbnail/plain/did:plc:5blrrzktebvkslsnxwlicxji/bafkreicg277bpxm5wh6u4sgmfao4j3a6waza7k4q7uo444zhs4t47mywya)
![Box 2. Generative agents and simulated worlds
Researchers note that ‘many of themost interesting research questions, such as the psychology ofworld leaders, the effects
of large-scale policy change, or the effects of large-scale events on the general public’ are ‘logistically infeasible’ to study in
the laboratory ‘with any realistic amount of resources’ [23]. In response, generative agents populating simulated worlds are
seen as promising research paths. For example, researchers could create generative agents based on the profiles of Palo
Alto residents and simulate how the community would respond to different pandemic interventionsv. Much of the technical
research on artificial agents acting in simulated worlds originates in fields beyond cognitive science, including computer science,
sociology, economics, political science, computational social science, as well as private industry [9,112–116].
Developers of these agent architectures have lofty ambitions. They believe that this technology can ‘test interventions and
theories and gain real-world insights’ [58], serving as ‘a high-fidelity platformfor policy outcome evaluation’ to enable ‘datadriven
policy selection’ [115]. Given these ambitions, validating that these models can generalize to the real world is imperative
[116], and some researchers caution that ‘current architectures must cover some distance before their use is reliable’
[58]. Yet, such validation faces a paradox: these models can only be validated against the ground truth of real-world data,
but their appeal lies in simulating scenarios where ground truth is not available. Some researchers [22] propose to meet this
challenge by identifying ‘the most proximal cases for which ground-truth data from human subjects is available’ and using
those cases to validate the simulation’s predictions ‘before turning the model to a domain in which no ground truth exists’.
However, there is currently ‘no consensus’ around how proximal is proximal enough [116].
Imp…](https://cdn.bsky.app/img/feed_thumbnail/plain/did:plc:5blrrzktebvkslsnxwlicxji/bafkreic3vp2uqan3apjzkzc4soubhzq47qtnzzpcwqncwbb566hprb4gma)
