iiinova — where your digital self keeps evolving

Jenna Russell @jennarussell.bsky.social · 08/10/2026

You may win some sort of award for reading so deep into that appendix 😂

Reposted by Jenna Russell

Vauhini Vara @vauhinivara.bsky.social · 06/10/2026

This is a really fascinating research demo, from the lab of Mohit Iyyer at the University of Maryland: It predicts the likelihood that the ideas contained in a piece of text are AI-generated versus human-generated. A couple of examples in this thread.

ideadetector.ai

IdeaLens: is this text human or AI?

We check a text's ideas and prose separately.

23916 0

Jenna Russell @jennarussell.bsky.social · 01/10/2026

Prior work focuses on synthetic data made for training or model collapse scenarios (a model trained on its own outputs). We instead study "wild" AI text: written by many different LLMs for human readers, sitting unlabeled in web crawls, and rising each month.

100 0

Jenna Russell @jennarussell.bsky.social · 01/10/2026

AI text does help when the target is AI text! On Cosmopedia, the loss-minimizing mix is over 90% AI at every budget. It also changes how models write. At 268M, AI-typical phrases per 1,000 words rise from 0.16 with no AI text to 0.54 at 4 AI tokens per human token.

100 0

Jenna Russell @jennarussell.bsky.social · 01/10/2026

Check your validation set! AI text is easy to predict, so every model we trained with added AI text got better on AI text. This hides potential harm: 95.5% of runs where AI text hurt human-text loss look better on a validation set that is 22% AI (like 2026 web).

100 0

Jenna Russell @jennarussell.bsky.social · 01/10/2026

Short on human data? Repeat it. At 20 tokens/param, one extra epoch of human text gives 2.3% lower C4 loss than adding the same number of new AI tokens. After eight extra epochs the gap is 6.3%.

100 0

Jenna Russell @jennarussell.bsky.social · 01/10/2026

If you care about loss on human text, filter AI text out of your pretraining data. Standard quality filters make this worse. In our Common Crawl audit, FineWeb's pipeline keeps AI documents 2.3× as often as human ones, and DCLM's keeps them 9.8× as often.

200 0

Jenna Russell @jennarussell.bsky.social · 01/10/2026

What does training on wild AI text cost? At August 2026's 31% AI share, training on the unfiltered crawl takes 1.6× the compute of training on its human part. At our forecasted 51% AI share for the end of 2028, it will take 3.0× the compute.

110 0

Jenna Russell @jennarussell.bsky.social · 01/10/2026

Fit on models up to 268M, it predicts the effect of AI text for models 3.6× larger with 41% lower error than the best existing law we tested, and about a fifth of Chinchilla's error.

100 0

Jenna Russell @jennarussell.bsky.social · 01/10/2026

Chinchilla values all tokens identically. Our law adds a saturating benefit and a harm term, so the value of an AI token (in human-token equivalents) can turn negative. Without AI text, it reduces to Chinchilla.

100 0

Jenna Russell @jennarussell.bsky.social · 01/10/2026

We find that for data-starved models (under ~10 human tokens per param), adding AI tokens lowers loss on human text. From Chinchilla-optimal (20) on, adding AI tokens raise loss almost immediately. In contrast, adding human tokens continues to lower the loss.

100 0

Jenna Russell @jennarussell.bsky.social · 01/10/2026

How much is an AI token worth? We fit scaling laws to find out! In August 2026, Pangram labels 31% of FineWeb-filtered web tokens as AI-generated, up from 10% in June 2024. We pretrained 800 LMs (19.9M–973M params) on mixes of human and AI web text to measure what it does to pretraining 🧵

1183 0

Jenna Russell @jennarussell.bsky.social · 28/09/2026

I’ll be presenting it at TADA next Monday and COLM next Wednesday if anyone wants to chat about StoryScope!

161 0

Reposted by Jenna Russell

Tuhin Chakrabarty @tuhinchakr.bsky.social · 28/07/2026

🚨 New paper on AI & Copyright Using @pangram.com full-text AI detection, we study self-published genre fiction on Amazon, 2023–26. Amazon is flooded w/ books with substantial AI text. They earn real 💰 & crowd out non-AI books. w/ @dhillonp.bsky.social @xinyuel.bsky.social & Jane Ginsburg (1/n) 🧵

67131 0

Jenna Russell @jennarussell.bsky.social · 25/06/2026

I'll be talking about the OG paper, what's happened since October, and what it means for journalism as a whole at our ACL oral presentation. Hope to see you there.

010 0

Jenna Russell @jennarussell.bsky.social · 25/06/2026

Check out the update and paper at ainewsaudit.github.io/

120 0

Jenna Russell @jennarussell.bsky.social · 25/06/2026

Unfortunately for @WSJopinion, @nytopinion, and @PostOpinions, slop-eds continue to rise. In June, roughly 6.5% of opinions are flagged for AI-use.

130 0

Jenna Russell @jennarussell.bsky.social · 25/06/2026

AI NEWS UPDATE We've added 200k+ recent news articles and 5k op-eds from Oct 2025 - June 2026. Local news is up to ~10.81% AI-generated.

162 0

Jenna Russell @jennarussell.bsky.social · 08/04/2026

Hard to say absolute best, but Claude definitely has a more unique writing style than the rest. Some of this could be the result of higher api pricing, so open models less likely to train off of large amounts of Claude generated text? Also partly Claude is better lol (imo)

110 0

Jenna Russell @jennarussell.bsky.social · 07/04/2026

Joint work with Rishanth Rajendhran, @miyyer.bsky.social, and John Wieting. Thanks also to UMD Clip for all the support!

390 0

Jenna Russell @jennarussell.bsky.social · 07/04/2026

We're releasing StoryScope code, all 10,272 prompts, 51,336 AI-generated narratives (~5k words each), and per-story features to support future work on narrative analysis & AI authorship. 📝 arxiv.org/abs/2604.03136 🔗 github.com/jenna-russe...

github.com

GitHub - jenna-russell/storyscope: This repo holds the code for the paper "StoryScope: Investigating idiosyncrasies in AI fiction".

This repo holds the code for the paper "StoryScope: Investigating idiosyncrasies in AI fiction". - jenna-russell/storyscope

2251 0

Jenna Russell @jennarussell.bsky.social · 07/04/2026

Style-based detectors' performance will vary as models evolve (GPT already cut the em-dash). Narrative structure is harder to humanize, changing it requires significant rewriting instead of post-hoc edits. StoryScope can be used as a more durable basis of authorship analysis.

1180 0

Jenna Russell @jennarussell.bsky.social · 07/04/2026

Narrative features are robust to stylistic editing. When we ran a subset stories over the LAMP protocol (e.g., rewriting cliches, purple prose) detection only dropped from 95.5% -> 93.9% macro F1. Surface-level 'humanization' doesn't fix structural tells.

1220 0

Jenna Russell @jennarussell.bsky.social · 07/04/2026

🟢 Gemini has the tidiest endings and bleakest settings (88% bleak). 🟠 DeepSeek likes to front-load crucial context (humans and others hold it until the end). 🟣 Kimi is the most generic, with few choices distinctive from other models.

1250 0

Jenna Russell @jennarussell.bsky.social · 07/04/2026

Each model has its own fingerprint 🔴 Claude keeps it cool - flat event escalations, follows traditional literary tropes, and likes epilogues, producing consistent, careful stories. 🔵 GPT is a gossip: uses rumors as plot devices, frames stories as decades-old retrospectives.

1250 0

Jenna Russell @jennarussell.bsky.social · 07/04/2026

The five AI models cluster together in narrative space, distinctly separated from human writing. Human stories are also rarer, based on neighbor distance in narrative space. 24.7% of human stories fall in the rarest 10% of the corpus, vs 7.1% of AI stories.

1605 0

Jenna Russell @jennarussell.bsky.social · 07/04/2026

Humans ... - Love nonlinear structures (flashbacks and time jumps) - Reference real texts and authors at ~2x the AI rate - Frames protagonists as more morally ambivalent (59% vs. 38%) - Diverse features: more characters, more dialogue, and more subplots (42%)

1417 0

Jenna Russell @jennarussell.bsky.social · 07/04/2026

What separates AI from humans? AI ... - over-explains its themes - narrators spell out the moral (77% vs 52% for humans) - AI favors clean, single-track plots (79% have no subplots) - over-writes the body; emotion is conveyed as 'tight chests & cold sweats' (81% vs. 38%).

1455 0

Jenna Russell @jennarussell.bsky.social · 07/04/2026

On simple XGBoost models, our narrative features hit 93.2% macro-F1 (0.96 AUPRC) for the human vs AI detection task, keeping 97% of the performance of our model using narrative and stylistic cues. Just 30 'core' narrative features capture the majority of the signal.

1230 0

Jenna Russell @jennarussell.bsky.social · 07/04/2026

We introduce StoryScope, a pipeline to extract interpretable narrative features (e.g., plot, character, revelation) across 60k+ stories, written by humans and 5 LLMs (Claude, GPT, Gemini, Kimi, DeepSeek) over the same ~10k prompts.

1240 0

Jenna Russell @jennarussell.bsky.social · 07/04/2026

Would you realize if the book you were reading was AI? What if it was humanized to remove AI-speak? We find that even without using stylistic cues (e.g., word choice or sentence structure) narrative choices alone give AI fiction away!

927775 0

Jenna Russell @jennarussell.bsky.social · 22/10/2025

Thanks to my amazing coauthors @markar.bsky.social, Destiny Akinode, @kthai1618.bsky.social, Bradley Emi, Max Spero and @miyyer.bsky.social and the support of UMD Clip lab and Pangram Labs

010 0

Jenna Russell @jennarussell.bsky.social · 22/10/2025

We will be continuously monitoring American news to keep up with how AI use changes over time. Follow along at 🌐 ainewsaudit.github.io

121 0

Jenna Russell @jennarussell.bsky.social · 22/10/2025

AI has been creeping into the news all of us read, often without any disclosure. We call for clearly defined standards for U.S. newsrooms: 1️⃣ Clearly define what counts as acceptable use of AI and publish these standards openly 2️⃣ Require AI-use attestations for all writers

1154 0

Jenna Russell @jennarussell.bsky.social · 22/10/2025

Many AI-written stories still contain authentic quotes. We hypothesize that people often use AI for editing or expanding on their human-written work. But with no disclosure, there's no way to tell for sure.

120 0

Jenna Russell @jennarussell.bsky.social · 22/10/2025

We also track how AI adoption has evolved over time: Among 10 veteran reporters we followed longitudinally, AI use rose from 0% pre-ChatGPT (2022) to >40% in 2025.

100 0

Jenna Russell @jennarussell.bsky.social · 22/10/2025

AI is disproportionately affecting news written in languages other than English. Roughly ~8% of English news is AI-generated, compared to 33% of non-English languages (primarily Spanish). Without disclosure, we cannot be sure whether AI is translating stories or writing them.

100 0

Jenna Russell @jennarussell.bsky.social · 22/10/2025

In NYT, WaPo & WSJ, opinion sections show 6.4× higher AI use than other sections, rising ~25× since 2022 (from ~0% → ~4%). AI use is concentrated among prominent guest authors: politicians, CEOs, and scientists.

142 0

Jenna Russell @jennarussell.bsky.social · 22/10/2025

Despite widespread use, transparency is basically nonexistent. Out of 100 AI-flagged articles we manually annotated, only 5 disclosed that AI was used and over 90% of outlets have no public AI policy.

130 0

Jenna Russell @jennarussell.bsky.social · 22/10/2025

AI use isn’t evenly distributed: 🗞️ Far higher in small local papers than national outlets 🌎 Especially common in Mid-Atlantic & Southern states 🏢 Largely Driven by ownership groups (e.g. Boone Newsmedia & Advance Publications) 🧭 Most concentrated in weather, tech, and health

134 0

Jenna Russell @jennarussell.bsky.social · 22/10/2025

We detect AI using Pangram, a model with a reported false positive rate of 0.001% on news text. We find that 5.2% of recent news Is completely AI-generated, with another 3.9% partially AI-generated. www.pangram.com/

pangram.com

Pangram Labs AI Detection

The most accurate technology to detect AI-generated content. Detects ChatGPT, Gemini, Meta AI, Claude, and more. Supports 20+ languages with 99.98%+ accuracy.

121 0

Jenna Russell @jennarussell.bsky.social · 22/10/2025

AI is already at work in American newsrooms. We examine 186k articles published this summer and find that ~9% are either fully or partially AI-generated, usually without readers having any idea. Here's what we learned about how AI is influencing local and national journalism:

55629 0

Reposted by Jenna Russell

Chau Minh Pham @chautmpham.bsky.social · 03/06/2025

🤔 What if you gave an LLM thousands of random human-written paragraphs and told it to write something new -- while copying 90% of its output from those texts? 🧟 You get what we call a Frankentext! 💡 Frankentexts are surprisingly coherent and tough for AI detectors to flag.

1369 0

Reposted by Jenna Russell

Shana Gadarian @sgadarian.bsky.social · 08/04/2025

International students will stop coming to American universities if their visas are going to be at risk. This will make our intellectual community poorer and also make tuition more expensive for domestic students.

7590163 0

Reposted by Jenna Russell

John Skiles Skinner @skiles.blue · 21/03/2025

There is a quasi-religion in Silicon Valley that views AI as godlike. This faith has always been parallel to Evangelical Christianity: salvation (transhumanism), the rapture (the technological singularity), and demons (Roko's Basilisk) Lately the AI faith has fully fused with Christian Nationalism.

10159461413 0

Reposted by Jenna Russell

Yixiao Song @yixiaosong.bsky.social · 12/03/2025

Introducing 🐻 BEARCUBS 🐻, a “small but mighty” dataset of 111 QA pairs designed to assess computer-using web agents in multimodal interactions on the live web! ✅ Humans achieve 85% accuracy ❌ OpenAI Operator: 24% ❌ Anthropic Computer Use: 14% ❌ Convergence AI Proxy: 13%

1115 0

Reposted by Jenna Russell

Yekyung Kim @yekyung.bsky.social · 05/03/2025

Is the needle-in-a-haystack test still meaningful given the giant green heatmaps in modern LLM papers? We create ONERULER 💍, a multilingual long-context benchmark that allows for nonexistent needles. Turns out NIAH isn't so easy after all! Our analysis across 26 languages 🧵👇

1155 0