iiinova — where your digital self keeps evolving

Andrew Trexler @atrexler.com · 24/04/2026

In our last analysis, we use simulations grounded in our observed data to investigate the bias-precision tradeoff. RM designs always offer superior power; this benefit declines as N-per-arm gets large. Yet RMs will be more accurate to true ATEs in expectation unless PO designs are very well powered.

Figure 6 from the paper, which shows how statistical power changes as sample size, true ATE, and true design effect change. RM designs always offer superior power but this benefit is less meaningful with large samples.

Figure 7 from the paper, which shows how absolute error changes as sample size, true ATE, and true design effect change. RM designs are more accurate in expectation for small samples, true ATEs, and true attenuations; PO designs are superior when samples, true ATEs, and true attenuations are large.

120

Andrew Trexler @atrexler.com · 24/04/2026

That said, we find initial evidence that the design effect increases as respondents are repeatedly exposed to RM designs within a single survey, and particularly when attitude-recall questions (asking Rs to recall their pre-treat attitude) are deployed. This has implications for omnibus surveys.

Table 5 from the paper, which shows nearly-monotonic increases in the design effect as respondents encounter more and more RM designs in a single survey.

120

Andrew Trexler @atrexler.com · 24/04/2026

Second, we observe few differences in the design effect by sample type (prob/nonprob), design type (between-groups/within-subject), and even by how close together repeated measures appear. RM design effects are no worse when putting pre-treatment measure, treatment, post-treatment back-to-back.

Figure 2 from the paper showing internal meta-analyses of the design effect on estimated ATEs. The estimates consistently fall around 20% attenuation regardless of sample & design type (except within-subject designs on Prolific).

Figure 4 of the paper, showing how the design effect changes by how close repeated measures are placed together. The change is mostly flat as distance separating measures goes from 0 to 20 survey items.

120

Andrew Trexler @atrexler.com · 24/04/2026

Our results: First, we find consistent evidence that RM designs attenuate ATE estimates by roughly ~20%. At the same time, PO designs are imprecise, requiring a large N for a reliable estimate. RM designs offer very large precision gains, by ~50% in our samples (note: N matters here).

Figure 5 from the paper, which shows histograms of the percentage change (from PO to RM) in the estimated ATE and standard error. The change in ATEs is clustered around 20% attenuations. The change in SEs is clustered around 50% reductions.

220

Andrew Trexler @atrexler.com · 24/04/2026

What we do: 1. Partially replicate CSP with 6 studies each fielded on 3 distinct samples (18 studies, 13163 participants, 78978 total obs). 2. Increase evidence on within-subject designs 43-fold. 2. Compare probability & non-prob samples. 3. Randomize how close repeated measures appear together.

Table 1 from the paper, summarizing each study's design, treatment, and control arms.

120

Andrew Trexler @atrexler.com · 24/04/2026

First, a quick preview of the findings: 1. Repeated measure designs attenuate ATE estimates by ~20% on avg relative to classic post-only measurement. 2. RM designs also provide massive gains in statistical power. 3. For most applications, power gains will trump ATE attenuation, so use RM designs.

Figure 3 of the paper. The figure displays the estimated ATE of each experiment under each design (bootstrapped). The point estimates for the repeated measure design are nearly all attenuated relative to the post-only design, but the confidence intervals are also much smaller.

150

Andrew Trexler @atrexler.com · 24/04/2026

Thrilled to announce my paper with @dianamejordan.bsky.social & sky-less Trent Ollerenshaw is now published open-access at @apsrjournal.bsky.social. In a (very) large replication & extension, we show that design effects of repeated measure experiments are nonzero but typically small. Thread:

Abstract for "New Evidence and Design Considerations for Repeated Measure Experiments in Survey Research" by Diana Jordan, Trent Ollerenshaw, and Andrew Trexler, published in the American Political Science Review. The abstract text reads: "We re-examine recent influential claims that repeated measure experimental designs offer large precision gains without biasing treatment effect estimates in survey research. We test these claims by experimentally varying the design of six classic political science experiments across three distinct large samples of U.S. adults (total N = 13,163). In contrast to prior evidence, we observe consistent attenuation of treatment effects in repeated measure designs. However, we show in simulations that this average design effect is small enough, and the precision gains large enough, that we recommend repeated measure designs for broad application—though (large-N) post-only designs may be preferable when research priorities include estimating the precise magnitude of a treatment effect. We additionally explore how several design considerations affect the bias-precision trade-off, such as within-subject versus between-groups designs, the relative separation of repeated measures within single surveys, and differences in respondent characteristics across sample types."

45228

Andrew Trexler @atrexler.com · 12/02/2026

These findings provide direct evidence that contemporary forms of political news are detrimental to the press's normative purpose in a democracy---informing the public. I also show that they affect norms too: exposure to "public interest" style coverage increases support for key democratic norms.

Figure 4 of "The Unequal Challenge of Learning from Under-Informative News." The figure shows the predicted post-treatment support for norm-breaking policies, conditional on baseline political engagement and treatment assignment. Support for norm-breaking declines among less politically engaged consumers following exposure to "public interest" style coverage.

110

Andrew Trexler @atrexler.com · 12/02/2026

Importantly, these effects are most severe for less politically engaged consumers, but persist even for highly engaged readers---especially when spending less time with an article.

Figure 2 of "The Unequal Challenge of Learning from Under-Informative News." The figure shows the predicted recall of key information under each time constraint, conditional on baseline political engagement and treatment assignment. The recall penalty is mostly restricted to less engaged individuals under no or a slight time constraint, though substantively large in the latter case. Under a severe time constraint, the penalty is large and consistent at all levels of baseline political engagement.

110

Andrew Trexler @atrexler.com · 12/02/2026

I show that readers recall less key info from the "under-informative" style articles than the "public interest" style. This learning penalty gets increasingly severe when engaging briefly, a common feature of media consumption today (i.e., "news snacking").

Figure 1 of "The Unequal Challenge of Learning from Under-Informative News." The figure shows the estimated average treatment effect on information recall following exposure to under-informative news under different time constraints. The estimate ATEs are consistently negative, with stronger effects under tighter time constraints.

110

Andrew Trexler @atrexler.com · 12/02/2026

These 4 styles are very common today. I contrast them with a rarer "public interest" style, which puts info relevant for democratic accountability front & center. I argue that most politics stories can be written with any of the 5 styles by making different decisions about info placement & language.

Table 1 of "The Unequal Challenge of Learning from Under-Informative News." The table displays 5 ways of writing a headline for the same story.

"Public interest" style: "Congress approves new military aid package for Ukraine in bipartisan votes."
"Conflict" style: "Congress approves new Ukraine funding, delivering Biden victory over GOP objections."
"Jargon" style: "Johnson pushes through Ukraine aid bill despite objections from Freedom Caucus."
"Prediction" style: "The House Speaker's push to approve new Ukraine funding might cost him his job."
"Clickbait" style: "Here's how the House Speaker got around far-right opposition to secure Ukraine aid."

110

Andrew Trexler @atrexler.com · 12/02/2026

First, I discuss 5 different styles of political news in terms of how they present info. "Conflict" style: focus on partisan battles. "Prediction": focus on forecasting. "Jargon": convey savviness via wonk language. "Clickbait": hide info behind a click. These 4 styles I call "under-informative."

An example clickbait headline: "Here's Why Trump Was Forced to Say He Has Only 'Concepts of a Plan.'"

An example conflict headline: "From Court to Congress to the Mideast, Trump Tallies His Wins."

An example jargon headline: "NOAA confirms investigation into RFK Jr. over dead whale carcass."

An example prediction headline: "Supreme Court seems skeptical of Trump's claim of absolute immunity but decision's timing is unclear."

110

Andrew Trexler @atrexler.com · 07/07/2025

Academics/adults with backpacks: I am in the market for a new backpack. Do you have a (work) backpack that you love? I have a North Face 'Hotshot' backpack that has served me faithfully for 10+ years but is finally wearing out, so I need to replace & want to consider options. Some parameters:

Image of a black 'Hotshot' backpack from The North Face against a grey background.

230

Andrew Trexler @atrexler.com · 04/07/2025

For several years, I have marked the 4th by reading Frederick Douglass's 1852 speech on the subject (a copy here: loveman.sdsu.edu/docs/1852Fre...), sometimes alone, sometimes aloud to family or friends. This year it feels especially poignant. A few passages that particularly stick with me today:

A portrait photograph of Frederick Douglass, circa 1879. Source: Wikimedia Commons.

24620

Andrew Trexler @atrexler.com · 17/04/2025

Not the main thing here, but I keep seeing this claim from the WH/Fox/etc. that egg prices are falling, and that is just objectively, observably false.

A graph of the price of a dozen eggs in the US over the past year. Data from the US government Bureau of Labor Statistics. The graph shows that average egg prices currently stand at $6.23, higher than at any point in the past year (and the highest since record-keeping began in the 1980s).

131

Andrew Trexler @atrexler.com · 03/04/2025

We also find limited evidence that ATE attenuation varies by how far apart repeated measure items are placed, meaning this design is suitable even for very short surveys/modules. Similarly, we find no evidence that respondent attention/recall alters the design effect.

Figure 3 showing the estimated ATEs of repeated measure experiments as the separation between pre- and post-treatment measures changes.

Figure A.3.2 showing the percentage of respondents who accurately or inaccurately recalled their level of observed opinion change in the repeated measure designs.

110

Andrew Trexler @atrexler.com · 03/04/2025

In contrast to CSP, we observe ~20% attenuation of ATEs under repeated measure designs. But the RM design offers much better precision, shrinking the SEs by ~50%, making hypothesis testing easier despite slight bias. Results hold across multiple sampling designs & experiment types.

Plot of bootstrapped estimated ATEs for different experiments, samples, and experimental designs.

121

Andrew Trexler @atrexler.com · 03/04/2025

New paper with @dianamejordan.bsky.social and sky-less Trent Ollerenshaw! We provide large-N tests of repeated measure designs in survey experiments, showing that they slightly attenuate ATEs relative to post-only designs, but provide large gains to precision. Thread below. Preprint: osf.io/q6czp

Figure 4 providing a graphical summary of the main results of the study: a slight attenuation of average treatment effects from a repeated measure design, and large reduction in the ATEs' standard errors (improved precision).

3329

Andrew Trexler @atrexler.com · 03/02/2025

The NYT homepage currently makes ZERO mention of a private citizen illegally shutting down an entire federal agency, which was created by an act of Congress and provides life-saving aid to millions around the globe in service of US national security.

A screenshot of the NYT homepage that fails to make any mention of a private citizen illegally closing an entire federal agency over the weekend.

021

Andrew Trexler @atrexler.com · 20/01/2025

This is better but still hides unnecessarily behind "scholars." The text of the Constitution is very explicit. "All persons born or naturalized in the United States, and subject to the jurisdiction thereof, are citizens of the United States." (The article correctly states this fact in paragraph 2.)

Push alert from The Washington Post that reads in part: "Trump will attempt to end birthright citizenship with an executive order. Scholars say the action would be unconstitutional."

182

Andrew Trexler @atrexler.com · 20/01/2025

As a deeply anti-democratic president assumes power today, I am begging news outlets to drop the prevaricating "will he or won't he, and can he succeed?" analysis frame. Instead, news orgs need to be crystal clear to audiences when his actions are incompatible with democracy & the rule of law.

44116

Andrew Trexler @atrexler.com · 13/01/2025

Some personal news: @hgraunke.bsky.social and I are moving to Wisconsin! I am thrilled to be joining the Political Science faculty at UW-Madison this Fall as an Assistant Professor. A huge, huge thank you to mentors, friends, and family in Durham and beyond. Thank you for sharing the journey.

A joyful picture of Andrew, Hannah, and Coconut. Andrew and Hannah appear happy and smiling; Coconut appears mildly put out.

11390

Andrew Trexler @atrexler.com · 23/10/2024

New-ish working paper! The media often uses conflict frames, clickbait, etc. for covering politics, ostensibly to make political news more appealing to low-attention voters. I show that demand for such entertainment-oriented news is instead driven by high-attention voters. Preprint: osf.io/59upd

Title and abstract for "The Paradox of Consumer Demand for Under-informative News," available at https://osf.io/59upd.

100

Andrew Trexler @atrexler.com · 24/07/2024

Anyway, for now, here is what I have. This is a modified measure with 9 labels and a "none of these" option, fielded on a non-prob Prolific sample (so caution please). These are the splits for Twitter use; I can also generate ones for FB, YouTube, Instagram, and TikTok if interested.

A series of graphs showing sample proportions of identification with ideological labels across self-reported Twitter usage.

100

Andrew Trexler @atrexler.com · 16/07/2024

In a follow-up conjoint experiment, we show that candidate signaling of alternative ideologies matters in both primaries and general elections. Voters self-identifying with the same alternative as a candidate are more likely to vote for them, just like when sharing liberal/conservative identities.

110

Andrew Trexler @atrexler.com · 16/07/2024

Our measure is particularly adept at understanding the policy attitudes of self-identified ‘moderates’ on the standard scale, who often choose the midpoint for reasons other than moderate or centrist policy preferences.

110

Andrew Trexler @atrexler.com · 16/07/2024

We find that self-identification with most alternative labels are indicative of consistent preferences on economic or social policy, and when considered in combination (e.g., ‘conservative environmentalist’ provides useful information about both heterogeneity and extremity of policy attitudes.

120

Andrew Trexler @atrexler.com · 16/07/2024

Many alternative labels cut across the partisan spectrum, with substantial numbers of self-identifiers affiliated with both Democrats and Republicans, including 'traditional,' 'environmentalist,' 'green,' and 'libertarian.' Others showed strong partisan leans, like 'nationalist' and 'progressive.'

120

Andrew Trexler @atrexler.com · 16/07/2024

Instead, we posed a novel question to a large Lucid sample (n=4,655): we asked people to self-identify with any of up to 14 different ideological labels that 'describe your political beliefs and attitudes.' Most people chose multiple labels, including many that were not 'liberal' or 'conservative'!

120

Andrew Trexler @atrexler.com · 23/05/2024

Tough call, but I think my favorite here is Ford graduating from UW in 1975... while he was president.

130

Andrew Trexler @atrexler.com · 09/05/2024

Importantly, we find these positive effects for both catastrophic and nuisance infrastructure issues. Effects are also similar for investigative reporting and (less resource intensive) contextual reporting. In either form, providing more information about risk boosts public spending support.

Figure 4 of the paper, showing little difference between investigative and contextual treatments.

112

Andrew Trexler @atrexler.com · 09/05/2024

We find that both types of info richness increases support for a pro-spending candidate, accountability for an anti-spending incumbent, and support for a costly bond measure for preventive infrastructure spending.

Figure 2 of the paper, showing positive electoral accountability and spending support effects from information-rich reporting.

Figure 3 of the paper, showing means on outcome variables by treatment condition.

131

Andrew Trexler @atrexler.com · 09/05/2024

We provide evidence for this with an experiment on two infrastructure systems (dam or sewer). We present respondents with a news article about infrastructure problems, and vary the level/type of info in the article: either barebones, or with investigative or contextual-thematic coverage (or both).

132

Andrew Trexler @atrexler.com · 09/05/2024

**Publication Alert!** "Local News Reporting and Mass Attitudes on Infrastructure Investment," from me & @mullinmeg.bsky.social in Political Behavior. We look at effects of information richness in local news on mass support for infrastructure spending. Open access! doi.org/10.1007/s111... Thread:

Figure 2 of the article, which shows positive information effects on electoral accountability and support for public infrastructure spending.

Figure 3 of the article, which shows average levels of support within each information condition for two mayoral candidates and a bond proposal.

22312

Andrew Trexler @atrexler.com · 01/12/2023

A good end to #AcWriMo for 2023. A few points left on the table of course, but lots of progress on lots of fronts. Looking forward to a whiskey (or several) very soon.

A table of goals and points for #AcWriMo 2023.

010

Andrew Trexler @atrexler.com · 22/11/2023

3 weeks of #AcWriMo down. Things are starting to come together and I am optimistic about knocking some big-ticket items off the list before the month is out.

A list of goals and tally of points for Academic Writing Month 2023.

000

Andrew Trexler @atrexler.com · 10/11/2023

10 days into #AcWriMo and my tally still looks sparse, but my writing pace has been good/steady and I am getting very close to tripling my points. Also, 6 weeks after submitting a manuscript it finally got assigned to an editor (you know where!), so I am celebrating(?) today.

000

Andrew Trexler @atrexler.com · 24/10/2023