iiinova — where your digital self keeps evolving

Noah Haber @whaleactually.com · 14/09/2026

For the curious: doi.org/10.1016/j.an... or free version on arXiv arxiv.org/abs/2004.04251 Comes complete with an R package to auto reveal assumptions hidden in your DAG to make you hate it/yourself

020

Noah Haber @whaleactually.com · 08/09/2026

And there's some bonuses in there. You want simulations???? Heck yeah you do. We needed some way to understand the whole Registered Revisions and policy process, plus bonuses like pre-writing our analysis code based on simulated data. How about a live demo webapp? Yup.

130

Noah Haber @whaleactually.com · 08/09/2026

Most importantly, what did the pilot journal editors (coauthors) think about the whole thing? What worked well, what was frustrating, and what went horribly wrong? We compiled our editorial team's thoughts so you can see for yourselves with (we think?) no sugar coating, good and bad alike.

130

Noah Haber @whaleactually.com · 08/09/2026

How about workflows and templates for integrating the Registered Revisions policy into standard editorial processes? Yeah we got that. You want infrastructure? You're covered. A whole webapp based system for participant management and tracking designed to work with normal editorial touchpoints.

130

Noah Haber @whaleactually.com · 08/09/2026

Part 2: the Pilot, with a crew of kickass journal editors and co designers. Unlike standard pilots confirming or adjusting a well-developed trial, we needed to build this plane as we were flying it. The "pathfinder" pilot was more co-development than test implementation. Everyone pitched in a TON.

130

Noah Haber @whaleactually.com · 08/09/2026

To estimate its impact, we are doing something out of the box: a "meta trial" Rather than one big trial with a bunch of journals or a hopeful meta-analysis of independent trials, we do a bit of both. One centralized design and infrastructure, but with flexible independent journal-based trials.

141

Noah Haber @whaleactually.com · 08/09/2026

... and that's where Registered Revisions comes in The Registered Revisions Meta Trial Pilot Report is OUT NOW!! osf.io/preprints/me... But new(ish) policy involves developing a new(ish) multi-headed trial design, workflows, infrastructure, piloting, mistakes, and a super team. And it's all here.

22117

Noah Haber @whaleactually.com · 08/09/2026

Most importantly, what did the pilot journal editors (coauthors) think about the whole thing? What worked well, what was frustrating, and what went horribly wrong? We compiled our editorioal team's thoughts so you can see for yourselves with (we hope) no sugar coating.

000

Noah Haber @whaleactually.com · 02/09/2026

<refresh> <refresh> uuuuuugh come onnnnnn <refresh> * human mods do your thing as long as you need, i appreciate you

110

Noah Haber @whaleactually.com · 02/09/2026

One might call that policy some kind of "registered revision" perhaps, and mayhap some folks went out and made a mega weird randomized meta trial situation, and allegedly a giant report describing the devlopment and piloting of such a thing is getting released shortly, who can say...

1174

Noah Haber @whaleactually.com · 13/08/2026

There are two kinds of people: those who see a serene landscape in this photo, and those who know. Source: sectionhiker.com/mt-washingto...

110

Noah Haber @whaleactually.com · 07/07/2026

This blog post shows how we made a *collaborative* reproducible manuscript workflow for SCORE papers It was not only possible, but by far the most *practical* option. The post discusses workflows for reproducible manuscripts, a demo R package for making them, and worked examples. Enjoy!

A chart showing a workflow for markup-style workflows. Data enters in from the left, and a script includes analysis code and narrative text all mixed into the same market script, which produces the final populated manuscript.

This image shows a workflow for placeholder-style reproducible manuscripts. In this case, data and analysis code are combined to produce analysis outputs. Separately, a manuscript text is written and collaborated on, containing placeholders for where analysis values will go. A script knits these two together, inserting analysis values where the placeholders are, producing the final populated manuscript.

140

Noah Haber @whaleactually.com · 02/04/2026

update: internal muffled scream

230

Noah Haber @whaleactually.com · 28/07/2025

It's sorta like someone smashed together a multi-center trial with a prospective meta analysis. If this works, it's a potentially game-changing way to do large scale policy evidence generation. And we have a good idea that it DOES work, because we've been piloting it with six journal partners.

100

Noah Haber @whaleactually.com · 28/07/2025

Hi folks! Want to reintroduce a thing I'm leading at @cos.io: the Registered Revisions (meta) Trial. This project about new peer review policy and a WILD new way of doing actionable evidence generation via RCTs. A LOT of RCTs. Now piloted and ready for the main stage, and looking for partners! 🧵👇

22113

Noah Haber @whaleactually.com · 14/03/2025

how it started / how it's going www.science.org/doi/10.1126/... www.aaup.org/news/cowardi...

140

Noah Haber @whaleactually.com · 25/02/2025

The practice of science is rejecting and challenging authority, not being obedient to it. Science is punk.

182

Noah Haber @whaleactually.com · 25/02/2025

aaaahahahahahahahaa this is the best

1220

Noah Haber @whaleactually.com · 25/02/2025

Absolutely not. This time calls for an array of approaches, righteous (and just plain right) fury included. This take is somehow simultaneously entirely predictable and surprising. It's profoundly demoralizing and ineffective, maybe harmful. www.science.org/doi/10.1126/...

4120

Noah Haber @whaleactually.com · 10/02/2025

The best I can offer as advice to the incoming generation of scientists. The most important one BY FAR is finding your people. The subtext for that is to also understand who will not be your people.

1134

Noah Haber @whaleactually.com · 03/02/2025

www.reuters.com/world/us/us-...

000

Noah Haber @whaleactually.com · 03/02/2025

Reuters is now covering the order for all CDC scientists to withdraw, take their name off of, or edit all existing publications in consideration if it has such evil words as "prenant people" This is research that is done and written. Straightforwardly censorship. www.reuters.com/world/us/us-...

153

Noah Haber @whaleactually.com · 02/02/2025

Can't be a plug if I'm shutting it down :) I make a honing guide for sharpening woodworking tools (chisels, plane blades and the like). www.deadedgedesign.com

3110

Noah Haber @whaleactually.com · 23/12/2024

This one's for those out there for whom this time of year can be rough. Here's a few things I made this year. Personally, I'll be doing some bike maintenance and a few small completable projects for the next week or so. What are you getting up to?

040

Noah Haber @whaleactually.com · 20/12/2024

In this particular case we are getting super mega meta, because we are ALSO pathfinding a new way to do this kind of evidence, involving centrally organizing and supporting a large number of semi-independent trials at many journals, ultimately resulting in a meta analysis.

120

Noah Haber @whaleactually.com · 15/10/2024

Playing with a chart idea for common replication rate measures (whether the original value is contained in the replication 95% CI and vice versa). Each effect size is rescaled to its counterpart effect size's CI so you can see the distribution, rather than a binary. Thoughts? note: fake data

241

Noah Haber @whaleactually.com · 30/09/2024

Yep. Money where my mouth is: for the Registered Revisions project, the primary outcomes of the meta-trial are time-to-event-based process outcomes. Research outcomes are secondary/tertiary. www.tandfonline.com/doi/full/10....

011