iiinova — where your digital self keeps evolving

errr pretend i remembered that I needed to account for clustering by respondent

They include only respondents who complete at least 3 of 12 learning phase problems (where the treatment group had AI assistance vs none for control). Data and code on Github, if you drop this exclusion criteria there is no effect on pass or skip rates between AI and control groups. So, eh?

1100

A Jay Holmgren @ajh.bsky.social · 06/10/2026

This is why I just automatically delete all of those "you're invited to join the Prestigious Honor Society" emails - I would never be part of a club that would have me as a member!

020

A Jay Holmgren @ajh.bsky.social · 05/10/2026

Just clicking around their OSF pages there are a number of weird inconsistencies, papers where they note that the replication is under-powered, then some findings within that paper are excluded due to low power, but others are included and marked as failing replication with no explanation I can find

081

A Jay Holmgren @ajh.bsky.social · 05/10/2026

Yeah, it seems like their method was to define a pretty strict criteria for "replication" and stick to it regardless of the valence of the result, but with a ton of inconsistency in the actual replication process.

140

A Jay Holmgren @ajh.bsky.social · 05/10/2026

Deviating from your own pre-registration with no explanation to label the original paper as "Not Replicable" based on an under-powered directionally consistent result in a different context is a stretch.

2161

A Jay Holmgren @ajh.bsky.social · 05/10/2026

Even conceptually, they're using data from a decade later - is that a replication, or is that testing whether the effect is durable over time with a different set of people? Social science claims aren't like physics, "migration can change voting patterns" is context-specific!

1212

A Jay Holmgren @ajh.bsky.social · 05/10/2026

Their pre-registration says they'll use n=848, and they therefore don't do a power analysis. But then in the actual replication they're down to n=224. They then say "This doesn't replicate" because p=0.19. But its directionally similar and they have one-quarter of the sample!

4172

A Jay Holmgren @ajh.bsky.social · 05/10/2026

The first "did not replicate" econ paper I looked at was their replication of aeaweb.org/articles?id=... But their replication with new data has a much lower sample size!

1150

A Jay Holmgren @ajh.bsky.social · 01/10/2026

totally insane (complimentary) season of tv

010

A Jay Holmgren @ajh.bsky.social · 28/09/2026

small email signature update

2200

A Jay Holmgren @ajh.bsky.social · 27/09/2026

In my experience customization demands aren't from CIOs, they're from clinical leadership who rarely internalize IT budgets so demands for customization are costless while adapting workflows will annoy your clinical staff, so easier to ask for customization then blame Epic.

000

A Jay Holmgren @ajh.bsky.social · 27/09/2026

On the Cosmos point it is not that Epic doesn't prioritize standardization, it is that it is impossible to force on customers who don't want it - Epic actively encourages customers to avoid customization and stay on their base Foundation model (good post at hitdoc.substack.com/p/epic-found...)

120

A Jay Holmgren @ajh.bsky.social · 27/09/2026

I'd disagree with the characterization they worsened interop in the service of their business. Not always perfect but on balance they've been net-positive, IMO, and certainly could've kept CareEverywhere closed but instead backed every major interop initiative pretty early.

100

A Jay Holmgren @ajh.bsky.social · 27/09/2026

I have no financial relationship to Epic, but I think boiling down the state of US interoperability to any single actor, or even the category of EHR vendors as whole, misses the point and elides a complex set of issues regarding incentive alignment, policy, and workflow!

110

A Jay Holmgren @ajh.bsky.social · 27/09/2026

(Demands for infinite local customization justified by “we’re special because we’re <UCSF/Stanford/Mayo/MGH>” is a huge barrier to the productivity gains you’d expect from IT, which standardizes workflows. But this is separate from interoperability!)

100

A Jay Holmgren @ajh.bsky.social · 27/09/2026

Funny enough, a key Epic v Cerner(Oracle) difference is Epic will push back against local customization requests more – they have a stronger “protect customers from themselves” culture, kind of an Apple vs Android analog. Cerner is MUCH more into the “selling ongoing services” business model.

100

A Jay Holmgren @ajh.bsky.social · 27/09/2026

This is all descriptive, tho I had a grad student who went into industry a few years ago and never published this but did a diff-in-diff on hospitals acquired by systems that primarily used Epic and switched EHRs to Epic - they increased data exchange both within and outside of their system.

100

A Jay Holmgren @ajh.bsky.social · 27/09/2026

Tl;dr: Interop is technically challenging but we know how to do it. Barriers are increasingly workflow, usability, clinician time, weak incentives, and non-HITECH sectors. Epics role is nuanced but they are certainly not a top barrier. In fact, “everyone use Epic” would probably increase interop!

100

A Jay Holmgren @ajh.bsky.social · 27/09/2026

(Note: lack of more prescriptive demands for interop was in hindsight probably a policy error. But HITECH was a hard job, tasked a tiny office to scale up and distribute tens of billions ASAP for stimulus + get the right balance of prescriptive requirements vs allowing innovation. Tough!)

100

A Jay Holmgren @ajh.bsky.social · 27/09/2026

This is mostly downstream of HITECH-era policy. HITECH paid provider orgs to adopt EHRs, but ONC set few prescriptive rules for exchange and built on a patchwork of pre-existing exchange networks. TEFCA is the attempt to link all those networks up now. www.thieme-connect.de/products/ejo...

thieme-connect.de

Please wait

100

A Jay Holmgren @ajh.bsky.social · 27/09/2026

Hospitals also don’t have strong macro incentives to share data since it reduces pt friction to move, clinicians don’t have strong micro incentives to rely on outside records for insured patients who won’t feel the cost of duplicative care especially when time costs of chart review more salient

100

A Jay Holmgren @ajh.bsky.social · 27/09/2026

Also other data elements e.g. SDOH data exchange is much more limited, though Epic hospitals do exchange more of it thanks in part to Care Everywhere academic.oup.com/jamia/articl... and my earlier linked JAMIA paper on the non-Core interoperability measures lagging core clinical data exchange use

academic.oup.com

Interoperability of health-related social needs data at US hospitals

AbstractObjective. To measure hospital engagement in interoperable exchange of health-related social needs (HRSN) data.Materials and Methods. This study co

100

A Jay Holmgren @ajh.bsky.social · 27/09/2026

Org-level connectivity issues do still exist but they’re mostly related to connectivity with orgs outside of HITECH funding, e.g. SNFs, behavioral health, etc.

100

A Jay Holmgren @ajh.bsky.social · 27/09/2026

Other EHR tasks (mostly documentation) crowd out limited clinician time to actually review outside records or even go look for them www.healthaffairs.org/doi/10.1377/... So the barriers are increasingly workflow related rather than technical connectivity via EHR vendor.

healthaffairs.org

Health Affairs Journal

100

A Jay Holmgren @ajh.bsky.social · 27/09/2026

In fact we’re rapidly hitting “too much data” rather than not enough, as half of physicians said outside records are just full of irrelevant data they lack time to parse through academic.oup.com/jamia/articl...

academic.oup.com

Health interoperability across phenotypes of family physician practices

AbstractObjectives. To inform initiatives to improve the interoperability of healthcare data, we described the experience of distinct phenotypes of physici

100

A Jay Holmgren @ajh.bsky.social · 27/09/2026

Cross-vendor exchange remains a barrier with 38% of family physicians said information from the same EHR vendor was very easy to use and only 8% said that about information from a different vendor. But Epic does best! jamanetwork.com/journals/jam...

100

A Jay Holmgren @ajh.bsky.social · 27/09/2026

But clinicians struggle to use them: fewer than 20% of physicians said outside data arrive automatically, easy to find, and easy to reconcile - though Epic is the only EHR vendor associated with better interop experience for at least 1 data element v Athena jamanetwork.com/journals/jam...

100

A Jay Holmgren @ajh.bsky.social · 27/09/2026

In terms of interop progress, the actual barriers are less attributable to a single org. We have lots of org-level connectivity! See rapid progress from www.healthaffairs.org/doi/full/10.... to academic.oup.com/jamia/articl...

healthaffairs.org

Health Affairs Journal

100

A Jay Holmgren @ajh.bsky.social · 27/09/2026

Plenty of Epic critiques are reasonable. They did lobby against info blocking rules in 2020. HIOs and hospitals report that EHR developers block information most often through price, though reports have fallen since 2015 doi.org/10.1093/jami.... They absolutely have their own incentives still!

doi.org

Information-blocking trends following regulatory action

AbstractObjective. To describe the prevalence of and trends in practices that interfere with the exchange of patient health information (potential informat

100

A Jay Holmgren @ajh.bsky.social · 27/09/2026

Epic was pretty supportive of FHIR development back to 2014, one of the first Carequality implementers, and one of the first TEFCA QHINs – and is currently by far the largest key facilitator of the current federal policy framework for interoperability!

100

A Jay Holmgren @ajh.bsky.social · 27/09/2026

Historically, Epic built an interop tool, Care Everywhere, years before other nat’l networks like CommonWell or Carequality, and then built connectivity to those networks into CE. Half of Care Everywhere transactions involve a non-Epic EHR according to their own stats.

100

A Jay Holmgren @ajh.bsky.social · 27/09/2026

Outside records are routinely available at the point of care more in Epic hospitals, hospitals report clinicians often use those records at higher rates, and only 25% of Epic hospitals report cross-vendor exchange as a major barrier vs 54% of others.

100

A Jay Holmgren @ajh.bsky.social · 27/09/2026

Empirically, Epic hospitals exchange more data! For 2025, 88% of hospitals that use Epic find, send, receive, and integrate outside records, vs 69% on other EHRs. 77% of Epic hospitals routinely integrate outside records without manual work vs 30% non-Epic.

bar chart of 2025 AHA IT hospital survey results

120

A Jay Holmgren @ajh.bsky.social · 27/09/2026

Not to say local build doesn’t cause problems. Two sites can record the same clinical data in different fields, and that makes data harder to standardize for research and quality measurement. But local build doesn't block interop – we're already using a standard, thats how the mapping happens.

110

A Jay Holmgren @ajh.bsky.social · 27/09/2026

But data exchange across organizations uses standards that are separate from local build. Post-21st century cures act, all certified EHRs have FHIR APIs and and a standard data set via USCDI. Hospitals with very different Epic builds still exchange records! Cross-vendor exchange does happen!

110

A Jay Holmgren @ajh.bsky.social · 27/09/2026

Customization costs and local build are real: user interface/UX changes, order sets, note templates, alerts, workflows. Every big health system regardless of EHR vendor does this, almost entirely from demand from clinicians/leaders. And this can get quite expensive given lots of local dev costs!

130

A Jay Holmgren @ajh.bsky.social · 27/09/2026

Since this is my real life academic expertise I felt compelled to comment. Local customization and interop are only somewhat related. Epic orgs report more data exchange and Epic has been at least somewhat interoperability-supportive historically. Some thoughts:

140

A Jay Holmgren @ajh.bsky.social · 24/09/2026

Saw this last night and it became the first piece of music to break me out of the new CRJ album on repeat in a week.

130

A Jay Holmgren @ajh.bsky.social · 22/09/2026

well hold on I would like to join a matt darling discord community.

051

A Jay Holmgren @ajh.bsky.social · 22/09/2026

Wild. I had ACA marketplace coverage for 2 years in my masters degree and never used it (healthy 20something years) but still valued the protection against catastrophic risk.

1130

Reposted by A Jay Holmgren

William B. Fuckley @opinionhaver.bsky.social · 20/09/2026

The thesis of this piece is that it’s fine to turn the country’s largest and best public university system into a diploma mill where the degree doesn’t mean anything because it’s more equitable.

2954650

A Jay Holmgren @ajh.bsky.social · 20/09/2026

just writing up my R01 resubmission hoping the administration doesn't know what "UCSF" stands for.

020

Reposted by A Jay Holmgren

Jamison Foser @jamisonfoser.bsky.social · 17/09/2026

our experiment with putting the dumbest fucking people in charge is unfolding with soul-crushing predictability

201456271

Reposted by A Jay Holmgren

lastpositivist.bsky.social @lastpositivist.bsky.social · 16/09/2026

It's genuinely fascinating that the US president keeps announcing that he just flat out doesn't know what trade is, he legit doesn't appreciate the concept of exchange and instead views economic activity through the lens of a dragon attempting to maximise its gold hoard to sit upon.

883852597

Reposted by A Jay Holmgren

Robert Fruchtman @fruchtose.bsky.social · 08/09/2026

If the Castro District became a landmark district, that means that a big chunk of land close to transit and specifically zoned for buildings up to 160' tall would no longer be eligible for HC-SF. Mandelman is effectively proposing to downzone the Castro District, especially for affordable housing.

1275

A Jay Holmgren @ajh.bsky.social · 06/09/2026

Unfortunately the hat goes crazy style

020

A Jay Holmgren @ajh.bsky.social · 03/09/2026

Occasionally funny but as an academic work some of the worst I've ever read, an almost baffling combination of hubris and ignorance from Graeber whose empirical claims have been thoroughly disproven.

000