iiinova — where your digital self keeps evolving

Sarah Griebel @sgriebel.bsky.social · 26/05/2025

This new study uses continued pretraining with historical documents on Qwen2.5, along with supervised fine-tuning and reinforcement learning for a more historically accurate CoT-tuned model. Cool methods! arxiv.org/pdf/2504.09488

arxiv.org

071

Reposted by Sarah Griebel

Ted Underwood @tedunderwood.com · 02/05/2025

New preprint from @lauraknelson.bsky.social, @mattwilkens.bsky.social, and myself tests different ways of simulating the past with LLMs. We don't fully answer the title question here—just show that simple strategies based on prompting and fine-tuning are insufficient. +

arxiv.org

Can Language Models Represent the Past without Anachronism?

Before researchers can use language models to simulate the past, they need to understand the risk of anachronism. We find that prompting a contemporary model with examples of period prose does not pro...

717956

Sarah Griebel @sgriebel.bsky.social · 26/11/2024

There are many ways to identify texts that seem ahead of their time. Our CHR 2024 paper asks which measures of textual precocity align best with social evidence about influence and change.

Abstract: Measures of textual similarity and divergence are increasingly used to study cultural change. But which measures align, in practice, with social evidence about change? We apply three different representations of text (topic models, document embeddings, and word-level perplexity) to three different corpora (literary studies, economics, and fiction). In every case, works by highly-cited authors and younger authors are textually ahead of the curve. We don't find clear evidence that one representation of text is to be preferred over the others. But alignment with social evidence is strongest when texts are represented through the top quartile of passages, suggesting that a text's impact may depend more on its most forward-looking moments than on sustaining a high level of innovation throughout.

1407