Sign in

Felix Holzmeister

@flxhlzmstr.bsky.social
1.3K followers 551 following 18 posts

Associate Professor @uniinnsbruck | Crowd-, Meta-, and Open Science | Behavioral and Experimental Economics and Finance | Lab Manager @econlab | @flxhlzmstr | www.felixholzmeister.com

PostsRepliesMedia
Reposted by Felix Holzmeister
Mark Rubin @markrubin.bsky.social · 14/11/2025
"p-Excuses" E.g., p = .056 is “marginally significant” or “close to being significant” 🔹 Most p-excuses are used in psychology. 🔹 The least p-excuses are used in health. Preprint: osf.io/preprints/me... #MetaSci #AcademicSky 🧪
Researchers face incentives to write up their empirical findings in a way that maximizes publication success. We analyze the language of journal articles and its association with statistical significance to explore questionable research practices at the stage of writing up articles, using 140,606 articles from health, biology, psychology, economics, and multidisciplinary journals over 32 years. For most disciplines, a higher share of non-significant main findings is associated with more hedging and negative striking words, fewer positive striking words, and fewer superlatives. We neither find evidence that authors upsell ambiguous results using sensational language nor that ambiguous results are written up less readably. Contrarily, articles with a higher share of statistically significant main findings are written up more sensationally. We find that emphasis on (marginal) statistical significance increases with the share of non-significant main findings, reflecting a dichotomized interpretation of p-values based on arbitrary thresholds. Particularly, p-excuses give the impression of statistical significance when the finding is actually not significant, consistent with the notion of 'spin'. This study provides empirical insights that might help researchers to self-reflect on writing up empirical findings. More training and fewer incentives to sell findings using sensational language and spin can help to improve academic writing.
42411
Reposted by Felix Holzmeister
Institute for Replication @i4replication.bsky.social · 30/10/2025
#GDRI_rep Update 9: We have been waiting a couple of months to make two announcements. A few journals were in the process of retracting GDRI article. The editors informed the authors and then lawyers got involved. Who knows how long it will take.
1127
Reposted by Felix Holzmeister
Lenka Fiala @fialalenka.bsky.social · 20/10/2025
Join us for the virtual Replication Games😍🤓 on Nov 13th! (You don't have to be in the UK - join us from wherever you are😉)
022
Reposted by Felix Holzmeister
Z-curve @replicationindex.com · 11/10/2025
🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥🔥
092
Reposted by Felix Holzmeister
Jörg Ankel-Peters @jrgptrs.bsky.social · 11/10/2025
Thought about scientific consensus recently? We have a new DP @i4replication.bsky.social that probes into the famous replication debate between Acemoglu, Johnson & Robinson (AJR) and Albouy - and how experts assess this debate. We find that they disagree. 1/8 www.econstor.eu/bitstream/10...
43511
Reposted by Felix Holzmeister
Lenka Fiala @fialalenka.bsky.social · 17/09/2025
Come joun us in Münster, it will be awesome😉😎
062
Reposted by Felix Holzmeister
Annie Whamond @anniewham.bsky.social · 18/09/2025
Excited to finally have my first paper officially published! "How Do Psychology Journals Handle Postpublication Critique?" is now online with AMPPS. Huge thanks to my supervisors @tomhardwicke.bsky.social and @simine.com and co-authors @bethclarke.bsky.social, N Moodie, S Schiavone, and R Thibault ☺️
doi.org
Sage Journals: Discover world-class research
Subscription and open access journals from Sage, the world's leading independent academic publisher.
14316
Reposted by Felix Holzmeister
Institute for Replication @i4replication.bsky.social · 27/08/2025
#GDRI_rep Update 8a: Major updates coming up over the next few weeks. 1st update: We reproduced "Parent–teacher meetings and student outcomes" some time ago. The paper was then retracted by the editors of the Euro Econ Rev. Additional studies are connected (see diagram) 🧵
11713
Reposted by Felix Holzmeister
Achim Zeileis @zeileis.org · 05/06/2025
📢 Come work with us! 🏛️ Faculty of Economics & Statistics, UIBK @econstatuibk.bsky.social‬ @uniinnsbruck.bsky.social‬ ‼️ Three job openings for Full Professor positions - Data Science - Microeconomics - Behavioral Economics Please share! #datascience #economics www.uibk.ac.at/de/info-beru...
uibk.ac.at
Aktuelle Ausschreibungen – Universität Innsbruck
None
12417
Reposted by Felix Holzmeister
AUSSDA @aussda.bsky.social · 26/05/2025
Are you a PhD-Candidate, Prae- or Post-Doc interested in open research practices and want to spend a few summer days in beautiful #Innsbruck? 👉 Then the #OpenScience Summer School at @uniinnsbruck.bsky.social might be the perfect match for you! www.uibk.ac.at/en/weiterbil...
uibk.ac.at
Open Science
Summer School
142
Reposted by Felix Holzmeister
Institute for Replication @i4replication.bsky.social · 09/05/2025
A re-analysis of Ciacci's (2024) "Banning the purchase of sex increases cases of rape: Evidence from Sweden" Journal of Population Economics reveals major issues. A year ago, reproducers Adema, Folke, and Rickne found coding errors driving the paper's key results. Let's unpack this in a 🧵
25024
Reposted by Felix Holzmeister
Johanna Rickne @johannarickne.bsky.social · 09/05/2025
🚨 REPLICATION REPORT UPDATE: One year ago, a tweet by John Holbein alerted me, @ollefolke.bsky.social, and @jopieboy.bsky.social to a paper with a shocking result about Sweden’s law criminalizing the purchase of sex.🧵
7386198
Reposted by Felix Holzmeister
Stephen Wild @stephenjwild.bsky.social · 24/04/2025
also
American chopper meme. Two men are yelling at each other.

YOU SHOULD REPORT THE MARGINAL EFFECTS.

I REPORTED THE COEFFICIENT VALUES. WHAT MORE DO YOU WANT?

PEOPLE HAVE A HARD TIME INTERPRETING THOSE VALUES.

MINUS MEANS LESS AND PLUS MEANS MORE. THAT'S GOOD ENOUGH!

INTERPRETING EFFECTS ON THE SCALE OF INTEREST IS NATURAL AND BENEFICIAL.
1182
Reposted by Felix Holzmeister
Olle Hammar @ollehammar.bsky.social · 04/04/2025
Great to see our comment now available online in the European Economic Review! (w/ @carlbonander.bsky.social, @nikljako.bsky.social, @flxhlzmstr.bsky.social, Gunther Bensch & Abel Brodeur @i4replication.bsky.social) 🙌
sciencedirect.com
“Try to balance the baseline”: A comment on “parent-teacher meetings and student outcomes: Evidence from a developing country” by Islam (2019)
Islam (2019) reports results from a randomized field experiment in Bangladesh that examines the effects of parent-teacher meetings on student test sco…
0115
Reposted by Felix Holzmeister
Jörg Ankel-Peters @jrgptrs.bsky.social · 03/04/2025
Our proposal for protocolled replicability & robustness assessments is out in Q Open. The main aim is to structure robustness reproductions, but also to extend the scope to external/construct validity as well as pre-specification checks. Comments welcome! academic.oup.com/qopen/advanc...
2207
Reposted by Felix Holzmeister
Anne Sophie Lassen @ansolassen.bsky.social · 31/03/2025
Do men increase their support for gender equality if they have daughters? Some papers suggest so! Join this meta-science project as a co-author, work with SOEP data and help us all learn more!
0113
Reposted by Felix Holzmeister
Lab² @labsquare.net · 31/03/2025
🔔 Call for Participation 🔔 Lab² is inviting researchers to take part in a multi-analyst study on the effects of having daughters on various outcomes. Join this metascience project as a co-author and gain the opportunity to work with SOEP data!
manydaughters.com
#ManyDaughters
Many Analysts
21713
Reposted by Felix Holzmeister
Institute for Replication @i4replication.bsky.social · 31/03/2025
Collaborators are looking for participants for a new multi-analyst study using SOEP data. For more information on registration, eligibility criteria, requirements, procedure, and timeline, please visit www.manydaughters.com. Deadline for registration is April 27
12112
Reposted by Felix Holzmeister
Institute for Replication @i4replication.bsky.social · 27/03/2025
#GDRI_rep Update 7: Retraction! Our comment of "Parent–teacher meetings and student outcomes: Evidence from a developing country" has led to the paper being retracted. Our comment is accepted as is. Short 🧵
12516
Reposted by Felix Holzmeister
Association for Psychological Science @psychscience.bsky.social · 14/03/2025
Many psychological scientists are now calling for a “heterogeneity revolution,” focused on uncovering individual and contextual differences in experimental outcomes.
psychologicalscience.org
When One Size Doesn’t Fit All
Many psychological scientists are now calling for a “heterogeneity revolution,” focused on uncovering individual and contextual differences in experimental outcomes.
1266
Reposted by Felix Holzmeister
Institute for Replication @i4replication.bsky.social · 14/03/2025
#GDRI_rep Update 5: We have a new report. We reproduced the paper entitled "Partisan Effects of Information Campaigns in Competitive Authoritarian Elections: Evidence from Bangladesh" by Ahmed, Hodler and Islam published at the Economic Journal. See below for links to report and authors' responses.
11813
Reposted by Felix Holzmeister
Institute for Replication @i4replication.bsky.social · 14/03/2025
Since we started mass reproducing GDRI, we have requested several replication packages. We have not managed to get a single new replication package as of today (excluding updated packages for one article at the request of editors).
0124
Reposted by Felix Holzmeister
Olle Hammar @ollehammar.bsky.social · 11/03/2025
In a new @i4replication.bsky.social report we replicate Islam (2019). Main issue: the randomized controlled trial was not randomized. And many other things on top of that. See full report for details. (w/ @carlbonander.bsky.social, @nikljako.bsky.social, @flxhlzmstr.bsky.social, Bensch & Brodeur)
2249
Reposted by Felix Holzmeister
Institute for Replication @i4replication.bsky.social · 11/03/2025
#GDRI_rep Update 4: We have a new report. We reproduced the paper entitled "Parent–teacher meetings and student outcomes" by Islam @ European Economic Review. See below for links to report and author's response.
12015
Reposted by Felix Holzmeister
Institute for Replication @i4replication.bsky.social · 09/03/2025
#GDRI_rep Update 3: The paper "Improving Hygiene and Sanitation through Parental Skill Training" by Asadul Islam, Umair Khalil and Tabassum Rahman was conditionally accepted at Economic Development and Cultural Change. We requested a replication package. It has now been withdrawn
13211
Reposted by Felix Holzmeister
Institute for Replication @i4replication.bsky.social · 03/03/2025
#GDRI_rep Update 1: I4R are now reproducing all published papers that use data from GDRI, or are closely related in other ways. Here is a first update on our work. 🧵
12720
Reposted by Felix Holzmeister
Nick Huntington-Klein @nickchk.com · 25/02/2025
After a long wait, the working paper for the Many-Economists Project: The Sources of Researcher Variation in Economics. We had 146 teams perform the same research three times, each time with less freedom. What source of freedom leads to different choices and results? papers.ssrn.com/sol3/papers....
papers.ssrn.com
The Sources of Researcher Variation in Economics
We use a rigorous three-stage many-analysts design to assess how different researcher decisions—specifically data cleaning, research design, and the interpretat
13345158
Reposted by Felix Holzmeister
Colin Camerer @cfcamerer.bsky.social · 24/02/2025
I4R doing the lord’s work. Hopefully these will be a collective wake up call to be more careful and transparent about complicated studies in applied micro (and elsewhere too)
1264
Reposted by Felix Holzmeister
Olle Hammar @ollehammar.bsky.social · 24/02/2025
Disregarding the sadness of this story, it’s been great working with this awesome team! @jrgptrs.bsky.social @fialalenka.bsky.social @jackfitzgerald.bsky.social @flxhlzmstr.bsky.social @nikljako.bsky.social @anderskjelsrud.bsky.social @kotsadam.bsky.social @essieconomist.bsky.social
1165
Reposted by Felix Holzmeister
Jack Fitzgerald @jackfitzgerald.bsky.social · 24/02/2025
As a coauthor on the AEJ:AE report, it’s a lot. Oh my god, It’s a lot. In that paper, outcomes are inconsistently handled both in the code and in the field, the paper’s data is connected to a bunch of other experiments, and we find irregularities in the raw survey files.
0286
Reposted by Felix Holzmeister
Nate Breznau @breznaunate.bsky.social · 24/02/2025
The MOOB that I host (massive open online bibliography) for #openscience and #metascience literature is approaching 2,000 articles. You can freely have it via @zotero.org www.zotero.org/groups/26082... 🧪
zotero.org
Zotero | Groups > Open Science Readings
Zotero is a free, easy-to-use tool to help you collect, organize, cite, and share research.
13010
Reposted by Felix Holzmeister
Institute for Replication @i4replication.bsky.social · 24/02/2025
After being alerted about possible misconduct, the I4R are reproducing published papers that use data from a specific NGO (GDRI). This thread releases the first 2 reports and provides more information about the work and responses/statements from authors journals and journals. 🧵
313759
Reposted by Felix Holzmeister
Institute for Replication @i4replication.bsky.social · 21/02/2025
This is based on our work. We will provide more information on Monday. Stay tuned and thanks Ray and Will for your help!
0217
Reposted by Felix Holzmeister
Institute for Replication @i4replication.bsky.social · 21/02/2025
Sunday 9PM ET / Monday 3AM CET / Monday 1PM Melbourne! See you all then :) #EconSky
0107
Reposted by Felix Holzmeister
Institute for Replication @i4replication.bsky.social · 19/02/2025
Abel: I am writing in my capacity as Chair of I4R. Last year, I was contacted by a researcher that alerted me of potential scientific misconduct in a set of papers all using data from a specific NGO.🧵
14722
Reposted by Felix Holzmeister
Journal of Behavioral and Experimental Finance @jbef.bsky.social · 23/01/2025
JBEF is happy to announce that we are now on LinkedIn, following a strategic decision to shift our social media activities away from X. We look forward to keeping you updated with new publications and news from the journal! More about the journal at www.sciencedirect.com/journal/jour...
elsevier.com
Finance Journals Ecosystem
We are excited the share with our community the launch of the Finance Ecosystem
0111
Reposted by Felix Holzmeister
Data Colada @datacolada.bsky.social · 07/01/2025
Did a police training improve officers' behavior during arrests? A forthcoming QJE paper with a flexible pre-registration and key p-values of p=.048 & p=.051 says yes. datacolada.org/122
datacolada.org
[122] Arresting Flexibility: A QJE field experiment on police behavior with about 40 outcome variables - Data Colada
A forthcoming paper in the Quarterly Journal of Economics (QJE), "A Cognitive View of Policing" (htm), reports results from a field experiment showing that teaching police officers to "consider differ...
14719
Reposted by Felix Holzmeister
Protzko @protzko.bsky.social · 23/12/2024
For those keeping track of replication rates across fields: experimental asset market research is looking at a cool 21.4% replication rate. From @chrhuber.bsky.social & @flxhlzmstr.bsky.social & Michael Kirchler www.econstor.eu/bitstream/10... #metasky #metascisky
1129
Reposted by Felix Holzmeister
Paolo Crosetto @paolocrosetto.bsky.social · 20/12/2024
How do public policies stack up? Is the impact of two policies larger than the sum of its parts? We look at this in the context of food - labels + price policies - & find they are *extremely* sub-additive. Also, labels do way better than prices. Paper: www.sciencedirect.com/science/arti... 🧵
54821
Reposted by Felix Holzmeister
David Schindler @david-schindler.de · 14/12/2024
A group of researchers has attempted to replicate our work on self-control and asset market prices published in the Review of Financial Studies (doi.org/10.1093/rfs/...) and not been able confirm our findings. See doi.org/10.2139/ssrn...
doi.org
Do experimental asset market results replicate? High-powered preregistered replications of 17 claims
Experimental asset markets provide a controlled approach to studying financial markets. We attempt to replicate 17 key results from four prominent studies, coll
412423
Reposted by Felix Holzmeister
Jan Marcus @janmarcus.de · 29/11/2024
🧵 Excited to share: "Replication Code Availability over Time and Across Fields", was just accepted by #EconomicInquiry! 🚀 Here's what @lukasfink.bsky.social and I uncovered about replication code availability for publications based on the German Socio-Economic Panel #SOEP 📈📉 doi.org/10.1111/ecin...
doi.org
Replication code availability over time and across fields: Evidence from the German Socio‐Economic Panel
Providing replication code is an inexpensive way to facilitate reproducibility. However, little is known about the extent of replication code provision. Therefore, we examine the availability of repl....
56017
Felix Holzmeister @flxhlzmstr.bsky.social · 25/11/2024
🤚
100
Reposted by Felix Holzmeister
Brian Nosek @briannosek.bsky.social · 19/11/2024
New in NHB: We systematically replicated 26 of 41 PNAS social science studies that used MTurk. The approach provides a proof-of-concept for using decision markets to select findings to replicate. Replication effect sizes were 45% of original effect sizes. www.nature.com/articles/s41...
nature.com
Examining the replicability of online experiments selected by a decision market - Nature Human Behaviour
This study finds that decision markets can be a useful tool for selecting studies for replication. For a sample of 26 online experiments published in PNAS selected by a decision market, the authors fi...
29852
Reposted by Felix Holzmeister
Brian Nosek @briannosek.bsky.social · 19/11/2024
Check out the paper for interesting details on the approach, and the findings with application of a variety of indicators of replication success. www.nature.com/articles/s41... Many thanks to @flxhlzmstr.bsky.social, Anna Dreber, and Magnus Johannesson for leading this collaboration!
nature.com
Examining the replicability of online experiments selected by a decision market - Nature Human Behaviour
This study finds that decision markets can be a useful tool for selecting studies for replication. For a sample of 26 online experiments published in PNAS selected by a decision market, the authors fi...
1113
Felix Holzmeister @flxhlzmstr.bsky.social · 19/11/2024
Thanks to a great team of co-authors, particularly to Anna Dreber and Magnus Johannesson for the great collaboration! (n/n)
030
Felix Holzmeister @flxhlzmstr.bsky.social · 19/11/2024
Our results offer proof of concept for decision markets, showing that they can be useful tools for deciding which replications to prioritize to facilitate efficient and effective resource usage. (6/n)
110
Felix Holzmeister @flxhlzmstr.bsky.social · 19/11/2024
Overall, 54% of the claims were replicated, and replication effect sizes averaged 45% of the original effect sizes. These findings are comparable to those of previous replication projects involving lab experiments. (5/n)
110
Felix Holzmeister @flxhlzmstr.bsky.social · 19/11/2024
Replicability, based on statistical significance in the same direction as in the original study, was 83% for the top-12 and 33% for the bottom-12. (4/n)
100