iiinova — where your digital self keeps evolving

Reposted by James E. Pustejovsky

Katie Mack @astrokatie.com · 06/10/2026

What is the IceCube neutrino observatory? Neutrinos are elusive particles that rarely interact, but can create flashes of light when they do. Most neutrino detectors are giant water tanks surrounded by light detectors. IceCube is kilometer-long strings of detectors embedded in the Antarctic ice.

16904243

Reposted by James E. Pustejovsky

Jack Wilkinson @jdwilko.bsky.social · 24/09/2026

The integrity of evidence synthesis is threatened by problematic randomised controlled trials. These may be fraudulent, or subject to critical errors. INSPECT-SR is a tool to assess trustworthiness of RCTs - out today: www.bmj.com/content/394/...

bmj.com

INSPECT-SR tool for assessing trustworthiness of randomised controlled trials

The integrity of evidence synthesis is threatened by problematic randomised controlled trials, where there are serious concerns about the trustworthiness of the data or findings. Such concerns could b...

12415

Reposted by James E. Pustejovsky

Robert Kelchen @robertkelchen.com · 22/09/2026

My department and college are searching for FOUR new tenure-track assistant professor colleagues! Please share, and application reviews begin October 15.

15747

James E. Pustejovsky @jepusto.bsky.social · 21/09/2026

Finally, we implemented all the estimators and bootstrapping methods in an R package: jepusto.github.io/metaselection/ The package fits the models and automates the bootstrapping process all in one call, and it supports easy parallel computing via the {future} package.

jepusto.github.io

Meta-Analytic Selection Models for Dependent Effect Sizes

Fits a flexible class of p-value selection models for meta-analysis and meta-regression models, providing standard errors and confidence intervals based on either cluster-robust variance estimators (i...

100

James E. Pustejovsky @jepusto.bsky.social · 21/09/2026

We implemented the extrapolation technique in the simhelpers R package (meghapsimatrix.github.io/simhelpers/) since it seems like it could be useful more broadly. More details in this blog post: jepusto.com/posts/Bootst...

meghapsimatrix.github.io

Helper Functions for Simulation Studies

Calculates performance criteria measures and associated Monte Carlo standard errors for simulation results. Includes functions to help run simulation studies, following a general simulation workflow t...

110

James E. Pustejovsky @jepusto.bsky.social · 21/09/2026

In the course of programming the sims, we implemented a novel technique for extrapolating bootstrap confidence interval coverage rates, inspired by some old work by Boos and Zhang (doi.org/10.1080/0162...).

100

James E. Pustejovsky @jepusto.bsky.social · 21/09/2026

We did a really gigantic simulation involving three different bootstrapping techniques and four different methods of constructing bootstrap CIs. That took some pretty serious computing...

100

James E. Pustejovsky @jepusto.bsky.social · 21/09/2026

The latter process is novel, as far as we know, and seems worth examining more. IMO we need think a lot more about *other* potential selection processes in the multivariate context, how robust our approach is under alternative selection processes, and how to identify one process versus another.

100

James E. Pustejovsky @jepusto.bsky.social · 21/09/2026

One was a multivariate selection process where the strength of selection depends on how many significant effects a study generates (with more significant effects leading to a higher chance that non-significant findings are also reported).

100

James E. Pustejovsky @jepusto.bsky.social · 21/09/2026

We evaluated the methods under two distinct selection processes. One was a favorable/charitable scenario where selection happens at the level of individual effect sizes, in which we would expect the marginal model to work well....

110

James E. Pustejovsky @jepusto.bsky.social · 21/09/2026

But that's just the high-level take-away. Several more technical bits kept us up rather late at night....

100

James E. Pustejovsky @jepusto.bsky.social · 21/09/2026

With that goal, we look at two different estimators for the model parameters and several strategies for uncertainty assessment. Based on a really big simulation, we find that marginal likelihood estimators and two-stage clustered bootstrap technique work well, and we recommend sticking to those.

100

James E. Pustejovsky @jepusto.bsky.social · 21/09/2026

This approach does involve a concession, in that we're not capturing the full data-generating process, but it seems like a worthwhile step towards thinking about more complicated, multivariate selection processes.

100

James E. Pustejovsky @jepusto.bsky.social · 21/09/2026

The main idea is to fit regular step-function selection models as if all the effect sizes were independent, so that we learn something about the marginal distribution of effect sizes. (Intuitively, this is similar to using GEE with an independent effects working model for longitudinal data.)

100

James E. Pustejovsky @jepusto.bsky.social · 21/09/2026

*Thrilled* (and relieved, TBH) to share our new article (with Megha Joshi @meghapsimatrix.bsky.social and Martyna Citkowicz) on marginal step-function selection models for meta-analyses involving dependent effect sizes, just out in Research Synthesis Methods: doi.org/10.1017/rsm....

161

James E. Pustejovsky @jepusto.bsky.social · 11/09/2026

To pool, or not to pool, that is the question; Whether 'tis nobler for the meta-analyst to lump All of their data into one long data.frame, Or to take stock of a sea of potential moderators And by controlling, learn about them...

static.klipy.com

Jussie Smollett Speaking To Dead Skull

ALT: Jussie Smollett Speaking To Dead Skull

030

James E. Pustejovsky @jepusto.bsky.social · 10/09/2026

[X] Strongly agree [ ] Somewhat agree [ ] Neither agree nor disagree [ ] Somewhat disagree [ ] Strongly disagree

020

James E. Pustejovsky @jepusto.bsky.social · 04/09/2026

This package is tremendously useful, full-featured, and thoughtfully designed. Great to see that it's hit v1.0.0! Congratulations!

0130

James E. Pustejovsky @jepusto.bsky.social · 01/09/2026

Our package for fitting meta-analytic selection models to data with dependent effect size estimates is finally, officially, available on CRAN. I've been working on this one with Megha Joshi @meghapsimatrix.bsky.social and Martyna Citkowicz for a few _years_ now!

283

James E. Pustejovsky @jepusto.bsky.social · 01/09/2026

Our package for fitting meta-analytic selection models to data with dependent effect sizes is finally, officially, available on CRAN. Been working on this with Megha Joshi (@meghapsimatrix.bsky.social) and Martyna Citkowicz for a few _years_ now.

034

James E. Pustejovsky @jepusto.bsky.social · 31/08/2026

I've been noticing (informally) that LLM-generated text on scientific topics often includes (hyped) claims about "the mechanism" behind whatever main finding. In mad-libs style, the Results section has a subset called "____ as the mechanism." Is this an established tendency?

010

James E. Pustejovsky @jepusto.bsky.social · 23/08/2026

Yes. Reviewers need this sort of option too, since a slop ms be passable on an editors quick read but clearly bs to a reviewer who knows the details and literature on the specific topic.

010

James E. Pustejovsky @jepusto.bsky.social · 29/07/2026

And also he's got this famously colorful source text, full of legendary, literally larger-than-life personalities, and ends up with a movie where none of the principles have any personality to speak of. Odysseus shows very little wit or charm, Calypso is drab, and Athena is milquetoast.

010

James E. Pustejovsky @jepusto.bsky.social · 29/07/2026

I left thinking that it was sad that Nolan has an ungodly amount of money shoot with IMAX and used it to make a film where like 75% of the footage is people getting shot, slashed, drowned, beat up, or otherwise killed. (The shots of the boat and the islands were nice though.)

111

James E. Pustejovsky @jepusto.bsky.social · 29/07/2026

For whatever reason I'm not seeing any Odyssey chatter in my feed, so I'm probably just late to the party...but I saw the Odyssey on IMax. It was loud, long, and rather tedious. The dialogue is more wooden than some famous horses you may have heard of...

100

James E. Pustejovsky @jepusto.bsky.social · 29/07/2026

Come work with my amazing colleagues (and me)! We have an open rank search in the Learning Sciences area, seeking scholars working at the intersection of learning sciences and AI (broadly construed). Details in the link. wisconsin.wd1.myworkdayjobs.com/UW_Madison/j...

wisconsin.wd1.myworkdayjobs.com

Assistant, Associate, or Full Professor of Learning Sciences – RISE-AI (Artificial Intelligence)

Current Employees: If you are currently employed at any of the Universities of Wisconsin, log in to Workday to apply through the internal application process. Job Category: Faculty Employment Type: Re...

021

James E. Pustejovsky @jepusto.bsky.social · 20/07/2026

🚨PSA: Sending me LLM-generated pdfs with "your" questions about my work / requests that I look over "your" new work is not a good way to impress me, persuade me to collaborate with you, or convince me to mentor you. I have enough robots in my life already, thanks very much.

040

James E. Pustejovsky @jepusto.bsky.social · 12/07/2026

What the heck is going on?!?!? (Watching on Telemundo but I don’t speak Spanish.)

120

Reposted by James E. Pustejovsky

Matthew A Kraft @matthewakraft.com · 24/06/2026

🚨 I'm hiring for a (Senior) Research Associate position to work with me on a broad research agenda focused on teacher labor markets. Great opportunity to be a part of the vibrant @annenberginstitute.bsky.social community of researchers at Brown University. Links in reply Please spread the word

23637

James E. Pustejovsky @jepusto.bsky.social · 15/06/2026

* Conventional selection models can be used to estimate distribution of sample sizes in un-selected population (using inverse-selection weighting, as you do for z statistics). It might be instructive to compare N distribution for unwritten studies to the N distribution implied by selection model.

000

James E. Pustejovsky @jepusto.bsky.social · 15/06/2026

* The A&K selection model specification is very specific in that selection is on two-sided p-value. As a further specification check, it might be worth examining models with one-sided p-values (e.g., where z < -1.96, z < 0, z < 1.96 have different selection probabilities).

200

James E. Pustejovsky @jepusto.bsky.social · 15/06/2026

And protocols might provide info about whether outcome domain is primary or secondary target of intervention.

100

James E. Pustejovsky @jepusto.bsky.social · 15/06/2026

Also perhaps instructive for understanding the associations between effects and log(N). If it's really all about advance planning, then minimum detectable effect sizes from protocols should fully capture the observed associations....

100

James E. Pustejovsky @jepusto.bsky.social · 15/06/2026

Other thing I'm curious about (apologies if I'm being a pest): * Are you able to obtain detailed protocols for the written-up studies? Comparing planned to achieved sample sizes might be instructive (as the model treats the planned sample sizes from unwritten studies as equivalent to achieved)...

200

James E. Pustejovsky @jepusto.bsky.social · 15/06/2026

Meta-analyses in econ typically work with 3+ effect estimates per sample, not unheard of to have 8+. Less so in education, but mean estimates per study is still above 2 (see doi.org/10.1017/rsm....).

cambridge.org

100

James E. Pustejovsky @jepusto.bsky.social · 15/06/2026

Thanks for the response! That makes sense to me, but the rates of multiplicity in your current dataset (e.g., 64 estimates from 40 studies reporting test scores) seem quite low compared to what I would have expected....

100

James E. Pustejovsky @jepusto.bsky.social · 15/06/2026

In papers that provide lots of estimates, how did you select point estimates and SEs to include in the models?

110

James E. Pustejovsky @jepusto.bsky.social · 15/06/2026

This is very interesting work. I'm curious about how you all dealt with effect multiplicity in your corpus of studies. I would think that typical papers report impact estimates from multiple specifications, across multiple outcomes within a domain, and potentially across multiple sites....

110

Reposted by James E. Pustejovsky

Peter Bergman @peterbergman.bsky.social · 12/06/2026

New NBER working paper. We meta-analyze 82 RCTs of low-cost parent programs in 20+ countries. We use data on unwritten RCTs from funder records, RCT registries, author queries, etc. to estimate a model that adjusts for publication bias and characterizes the effect distribution for unwritten studies.

$NBER working paper. Paper title: Characterizing the File Drawer: Evidence from a Meta-Analysis of Parent-Interventions Around the World Abstract: We conduct a meta-analysis of 82 randomized controlled trials across more than 20 countries to estimate the effects of low-cost, remote parental engagement interventions delivered through text messages, phone calls, and apps. We estimate a joint likelihood function that incorporates both written studies and unwritten studies identified through trial registries, funder records, research labs, evidence clearinghouses, and other sources. By also recording sample sizes for unwritten studies, the model estimates the distribution of standard errors, identifies write-up probabilities conditional on significance, and characterizes the file drawer by estimating effect distributions for written \textit{and} unwritten studies. Bias-corrected effects are 0.05 SD for test scores, 0.07 SD for grades, 0.05 SD for attendance, and 0.03 SD for enrollment. In the best-identified domain, test scores, statistically insignificant results are still written up at high rates. We also find that larger studies tend to estimate smaller latent effects, which could indicate that true effects are correlated with study precision, violating a common meta-analysis assumption. In smaller-sample domains, our approach helps identify selection probabilities by anchoring the absolute write-up rates. Finally, we estimate the value of additional RCTs to inform adoption decisions. Any single study estimate is unlikely to dissuade adoption because parent interventions have high marginal value of public funds. Instead, future research is most valuable when it can explain heterogeneity across settings.$

12612

James E. Pustejovsky @jepusto.bsky.social · 09/06/2026

I reviewed a manuscript (for the same journal, I think) that had exactly these problems too. Which maybe makes sense, since if someone is going to do this, then why not do it several times?

030

James E. Pustejovsky @jepusto.bsky.social · 19/05/2026

What (TF?!??!) university did this????

110

James E. Pustejovsky @jepusto.bsky.social · 09/05/2026

Is it crass to brag that our Ed building is a clear exception? hga.com/projects/uni...

hga.com

University of Wisconsin - Madison - HGA

020

Reposted by James E. Pustejovsky

Jerry Chen @jcsalterego.bsky.social · 08/05/2026

:( :( :( more like cantvas

15773110

Reposted by James E. Pustejovsky

Wolfgang Viechtbauer @wviechtb.bsky.social · 27/04/2026

A new version (5.0-1) of the metafor package has been released on CRAN. It includes some smaller updates, including more ways to visualize prediction intervals / distributions. Further details here: www.metafor-project.org/doku.php/new... #Rstats #MetaAnalysis

A forest plot with various methods for visualizing the prediction interval / distribution at the bottom.

18527

James E. Pustejovsky @jepusto.bsky.social · 22/04/2026

Or would that depend on what else you might do?

010