iiinova — where your digital self keeps evolving

Dᴜsᴛɪɴ Sᴛᴏʟᴛᴢ

@dustinstoltz.com

239 followers 285 following 10 posts

Sociologist | Computational Social Science | Culture and Cognition www.dustinstoltz.com Mapping Texts: www.textmapping.com Mastodon: fediscience.org/@dustinstoltz

Posts Replies Media

Dᴜsᴛɪɴ Sᴛᴏʟᴛᴢ @dustinstoltz.com · 13/02/2026

Here's a little working paper with Marshall Taylor and Sanuj Kumar: Selecting Language Models for Social Science arxiv.org/abs/2601.10926 Thoughts welcome!

Currently, there are thousands of large pretrained language models (LLMs) available to social scientists. How do we select among them? Using validity, reliability, reproducibility, and replicability as guides, we explore the significance of:(1) model openness, (2) model footprint, (3) training data, and (4) model architectures and fine-tuning. While ex ante tests of validity (i.e., benchmarks) are often privileged in these discussions, we argue that social scientists cannot altogether avoid validating computational measures (ex-post). Replicability, in particular, is a more pressing guide for selecting language models. Being able to reliably replicate a particular finding that entails the use of a language model necessitates reliably reproducing a task. To this end, we propose starting with smaller, open models, and constructing delimited benchmarks to demonstrate the validity of the entire computational pipeline.

093

Dᴜsᴛɪɴ Sᴛᴏʟᴛᴢ @dustinstoltz.com · 02/02/2026

New paper out with Marshall Taylor and @olizardo.bsky.social: Functionaries: A Distributional Approach to Institutional Analysis Instead of institutions as things that contain people, we suggest institutions as expertise distributed across people. onlinelibrary.wiley.com/doi/10.1111/...

This paper outlines a distributional approach to institutional analysis, reconceptualising institutions as distributions of knowledge and activity across people. We argue that institutionalisation and institutional change are best understood by focussing on actors with the requisite knowledge and motivation to keep institutional patterns going, fix them when they go awry, or transform them when required, here called functionaries. The distributional approach allows us to distinguish between two main types of institutional change often conflated in the literature: Content-based and formal change. Content-based change, the one most often discussed, involves the importation, recombination, or expansion of specific patterns of activity. In contrast, formal change, often neglected in the literature, refers to shifts in the distribution of knowledge and activity, leading to dynamics of centralisation and decentralisation of institutional patterns. In this way, the distributional approach highlights the role of functionaries in both institutional stability and change, providing a micro-level perspective on institutional dynamics.

1146

Dᴜsᴛɪɴ Sᴛᴏʟᴛᴢ @dustinstoltz.com · 24/11/2025

Marshall Taylor and I just signed a contract to write the 2nd edition of "Mapping Texts" (www.textmapping.com) Inter alia, we'll include a couple more chapters on "generative" language tech* If you, or someone you know, has used such tech in social scientific investigation, let us know!

The image includes a mock up for the book "Mapping Texts: Computational Text Analysis for the Social Sciences" with a text box at the top saying "Second edition in the works!" and another text "With 100% more Al*." But this is not referring to artificial intelligence, but rather to an artist that is halfway to an EGOT: Weird "Al" Yankovic. An image of Homer Simpson is on the phone, presumably hearing about this new edition, and is so excited he exclaims "Whoo-hoo!" with a triumphant fist pump.

1152