iiinova — where your digital self keeps evolving

Mark Rubin @markrubin.bsky.social · 27/09/2026

New "Research Handbook on the Replication Crisis" Edited by David Trafimow with 17 chapters covering issues such as results-blind publishing, preregistration, theory development, and statistical inference. www.e-elgar.com/shop/gbp/res...

083

Mark Rubin @markrubin.bsky.social · 12/09/2026

"It is, therefore possible that in the light of the continuing violence in Northern Ireland both Protestants and Catholics may find that they can acknowledge their common heritage without threatening their strong group allegiances."

taylorfrancis.com

Dimensions of Social Identity in Northern Ireland | 10 | Changing Euro

Academic and official commentaries generally use the terms “Protestants” and “Catholics” to refer to the two communities in Northern Ireland. However, Whyte

132

Mark Rubin @markrubin.bsky.social · 12/09/2026

Stephen John (2026) considers the complexities of "Transparency in Science." www.taylorfrancis.com/chapters/oa-...

Conclusions The role and value of transparency in science is a vast topic. I have only scratched the surface. However, I do hope to have established three key claims. First, that the very same features of transparency which may make it attractive as a response to worries about “problematic values” in science—that it makes scientists act and reason as publics understand and appreciate—make it problematic as a response to worries about epistemic bad practice—because the public may just be confused on what counts as epistemic good practice. Second, that any account of what sorts of transparency are required to handle “problematic values” needs to avoid assuming that the influence of non-epistemic values is always a matter of conscious choice. Third, transparency may be a blunt tool for tackling mis- and dis-trust. These claims have important practical implications, because they suggest we should not always cheer calls for transparency. They also have important theoretical payoffs; clarifying why we value transparency has implications for how we understand trust and the role of values in justification. We might gain from being more transparent about the value of transparency.

371

Mark Rubin @markrubin.bsky.social · 12/09/2026

"Staff members at the agency, who spoke on the condition of anonymity out of fear of retaliation, are concerned that the changes will lead to further cuts for work that has long been core to the agency’s mission: discovery-focused research, led by the curiosity of scientists." #AcademicSky #MetaSci

nature.com

NSF moves to overhaul funding approach in line with White House priorities

The major funder of basic science has not said how its new initiatives will be funded — but staff worry streamlined approach will further squeeze cash for core sciences.

198

Reposted by Mark Rubin

λ³🎲 @cubiclogic.bsky.social · 11/09/2026

Βrexit dividends The north will decline further without international students bsky.app/profile/mark...

171

Reposted by Mark Rubin

Joe Bak-Coleman @jbakcoleman.bsky.social · 11/09/2026

I stand firmly by my assertion that metascientists should have seen this capture of science coming, but there is no reasonable way they can reasonably delude themselves into thinking the new nsf announcements are anything other than epistemic capture. www.science.org/content/arti...

science.org

Science’s reform movement should have seen Trump’s call for ‘gold standard science’ coming, critics say

Efforts to improve the rigor of research may have unwittingly handed the administration a way to attack science

29038

Mark Rubin @markrubin.bsky.social · 08/09/2026

"By stripping away many of the elements of Popper’s philosophy of critical rationalism that make his falsificationist theory of science a rich account of scientific inquiry, the authors are left with an impoverished and rather unsophisticated account of the hypothetico-deductive method." #PhilSci

142

Mark Rubin @markrubin.bsky.social · 07/09/2026

Negative Knowledge: "Knowledge about how to get knowledge" “Whilst the contributions of statistical and methodological experts were widely valued, the pursuit of negative knowledge was deeply unappealing to some interviewees." New work by @tayacollyer.bsky.social doi.org/10.1016/j.so...

Abstract
Population health research involves multiple disciplines including epidemiology, economics, biostatistics, health promotion, and social sciences. Disciplinary differences are widely acknowledged but rarely analysed, leaving the nature and source of interdisciplinary tension underspecified. Drawing on the concept of epistemic cultures and interviews with 45 international researchers sampled from a bibliometric network of health equity scholars, I investigate whether distinct epistemic cultures exist within population health research, and identify specific features underpinning reported interdisciplinary tensions. Analysis identified four distinct knowledge types: knowledge about society, knowledge about disease, knowledge about behaviour, and knowledge about how to get knowledge (‘negative knowledge’). These were associated with particular epistemic virtues, conceptualisations of health, and long-term goals. Three corresponded with epistemological styles previously described by Lamont; the fourth, directed at the apparatus of inquiry itself, fell outside this framework. What interviewees described as 'disciplinary differences' comprised a connected set of tensions across knowledge types, classification strengths, collection codes, and tolerance for complexity. Evaluative terms such as 'new', 'useful', and 'progress' did not carry consistent meanings across cultures, contributing to frustration and misrecognition in interdisciplinary collaboration. Written academic genres were a significant source of tension, reflecting the ways journals embed and reproduce disciplinary norms. The epistemic cultures identified were not independent: epidemiology's paradigmatic dominance exerts normative force, shaping the questions researchers ask and the form of acceptable answers. These differences have substance — they are specifiable, empirically grounded, and analytically decomposable — and they have consequences, for the questions researchers ask, the knowledge they prod…

141

Mark Rubin @markrubin.bsky.social · 07/09/2026

“Our results show that it is important to understand p-hacking in the context of the selection environment that produced it. That p-hacking could mitigate the effects of publication bias under extreme selection also hints at a self-correcting tendency in the scientific process.”

This paper studies the effects of p-hacking on the bias of published estimates when papers with statistically significant results are selectively published. We show that fast p-hacking---actions that lead to large changes in p-values---always exacerbates the bias from selective publication. On the other hand, slow p-hacking---actions that lead to small changes in p-values---exacerbates bias when selection is weak, but mitigates it when selection is strong. In a model featuring both types of p-hacking, we show that a normality assumption identifies the true distribution of effects as well as the counterfactual mean that would obtain under selective publication without p-hacking. Applying the model to meta-analyses on the effects of behavioral nudges and development aid, we find suggestive evidence that both mitigation and exacerbation can arise in practice.

252

Mark Rubin @markrubin.bsky.social · 05/09/2026

"This study examined the determinants of public trust in scientific findings and found that the communication of OS [open science] practices in layperson texts plays no substantial role in shaping trust." Open Access: doi.org/10.1177/1075...

Abstract
Public trust in science may depend not only on the quality of the research but also on laypersons’ attitudes toward the underlying topic. Although Open Science practices are meant to signal credibility, their influence on laypersons remains unclear, especially for scientifically controversial topics. In this preregistered Austrian study (N = 1,083), general trust in science and myside bias strongly predicted trust in certain popular-scientific summaries, while explicit Open Science cues did not show an effect. Exploratory analyses showed that participants’ optimistic conceptions of science were also linked to higher trust. Overall, biased beliefs appear more influential than transparency-related cues for trust in science.

076

Reposted by Mark Rubin

labfolio.bsky.social @labfolio.bsky.social · 03/09/2026

How does the metascience community think about the Contribution Roles Taxonomy (CRediT), in use at various prominent publishers? Is it any good? Does it miss any big categories of work? www.elsevier.com/researcher/a...

credit.niso.org

CRediT

Contributor Role Taxonomy

011

Mark Rubin @markrubin.bsky.social · 02/09/2026

The Journal of Trial and Error is looking for a new Editor with an interest in meta-research, open science, or academic publishing. More info: www.linkedin.com/posts/were-l...

096

Mark Rubin @markrubin.bsky.social · 02/09/2026

“There is no published study comparing time-to-publication for registered reports against standard articles." "If anyone is interested in trying to collect this data more systematically for political science, please let me @aaronerlich.bsky.social know.”

Here is what I find genuinely strange. That claim that Chambers and Tzavella make, the one I made in July, and the one every advocate of the format concedes has never been measured. There is no published study comparing time-to-publication for registered reports against standard articles. The literature has measured the format’s quality (reviewers rated registered reports higher on all nineteen criteria tested), its rate of null results (about 44 per cent of registered-report hypotheses supported, against roughly 96 per cent of results in standard psychology articles), its citations, and how many journals offer it. Nobody has measured the clock. The nearest work asks researchers what they think the cost is: Sarafoglou and colleagues surveyed 355 researchers on preregistration and got “better science but more work”; Imai and colleagues surveyed 519 experimental economists and found that more than 80 per cent know what a registered report is, most approve of them — and adoption is near zero. Neither asked anyone for a date.

253

Mark Rubin @markrubin.bsky.social · 02/09/2026

“To err is science.” Stephan Guttinger reviews Douglas Allchin's new book: "Toward a Philosophy of Error in Science" doi.org/10.1093/bios...

1215

Mark Rubin @markrubin.bsky.social · 01/09/2026

"What is clear is that more research is needed to determine when, if ever, MC [multiplicity control] is warranted and under what circumstances. In the current systematic review, we are interested in how empirical researchers utilize MC within their research." #stats

050

Reposted by Mark Rubin

Berna Devezer @devezer.bsky.social · 01/09/2026

Been reflecting some more on how for a decade+ there's been a whole outcry about how replications weren't published. Part of it is based on misleading data as it's now made clear (see the thread I'm replying to). But another part was that standalone replication studies weren't being published.

2243

Reposted by Mark Rubin

Matt Makel @mattmakel.bsky.social · 31/08/2026

I made a mistake. And because of that mistake, I no longer have confidence that results that I reported in several research papers are an accurate reflection of replication rates. 1/

19430

Mark Rubin @markrubin.bsky.social · 30/08/2026

Changes in economics research from 1998 to 2019: 🔸️ "Pre-analysis plans, power analyses accompanying null results, and corrections for multiple hypothesis testing remain uncommon." 🔸️ "Articles now report more tables and statistical tests." doi.org/10.1016/j.eu... #Economics #MetaSci #Methodology

Abstract
We map the evolution of economics research from 1998 to 2019 to assess progress and identify remaining challenges. Using text mining of 30,675 full-text articles and an in-depth analysis of 578,132 statistical tests from 3746 articles across a broad range of journals, we document changes in research practices over time. We highlight the influence of the credibility revolution and the open science movement. Awareness of reproducibility has increased modestly, largely driven by journal adoption of data and code sharing policies. At the same time, reliance on non-public data has grown, especially among authors at top-five universities, posing new challenges for transparency. Sample sizes have increased substantially, while reported effect sizes have declined, consistent with reduced exaggeration and potentially more policy-relevant estimates. Econometric methods and causal inference practices have become more sophisticated, and articles now report more tables and statistical tests. However, pre-analysis plans, power analyses accompanying null results, and corrections for multiple hypothesis testing remain uncommon. Article length has remained stable or declined slightly, and authors from top-five universities continue to represent a large and stable share of publications in top journals. Our findings provide a baseline for tracking future developments and for guiding efforts to further strengthen the credibility and transparency of economics research.

030

Mark Rubin @markrubin.bsky.social · 30/08/2026

"Our findings reveal that there are clear limits to what Republican partisans of color are willing to endorse ideologically, even in a highly polarized era." By Emily Ortiz, @rbalhambra21.bsky.social, @efrenpolipsy.bsky.social et al. doi.org/10.1017/rep.... #SocialPsyc polisky #PolPsych

Abstract
Non-trivial shares of people of color (PoC) have long identified as Republicans, and many of them helped cement Donald Trump’s 2024 re-election. How durable was this shift toward the political right among some PoC? We test two claims in this brief research note. First, consistent with polarization literature, we claim that partisanship primed three conservative postures among Republican PoC during the 2024 campaign: authoritarianism, social dominance orientation, and system justification. Second, aligning with work on expressive survey responses, we argue that these postures faded significantly once Trump’s agenda produced visibly harmful consequences for PoC (e.g., deportation raids; weakened civil rights). Leveraging a three-wave panel of Black, Latino, and Asian adults (N = 3,626), our pre-registered analyses indicate Republican partisanship in June 2024 heightened authoritarianism, social dominance, and system justification among Black, Latino, and Asian adults in December 2024. These patterns were fleeting, with partisanship’s impact on these postures withering by July 2025. We find scattered evidence that these postures intensified PoC’s Republican partisanship during this period, implying our results reflect campaign effects rather than an unerring endorsement of these right-wing inclinations among Republican PoC.

070

Mark Rubin @markrubin.bsky.social · 30/08/2026

"If persuasive devices cannot be clearly distinguished from QRPs, their use may undermine the integrity of scientific communication." New work by @rritabajraktari.bsky.social and Marcus Munafò doi.org/10.36850/b36...

Abstract
Scientific communication is often assumed to prioritize objectivity, yet persuasive strategies may subtly distort research interpretation. While the spread of errors in research is often attributed to methodological or statistical biases (Świątkowski & Dompnier, 2017), rhetorical strategies are less often considered questionable research practices, although they can be (Corneille et al., 2023). This study empirically investigates how authors and external readers perceive persuasiveness across different sections of scientific articles (Abstract, Introduction, Methods, Results, Discussion). Using a mixed-methods approach, we surveyed 15 authors of selected articles and 42 external researchers, asking them to rate paragraphs on persuasiveness and informativeness. No significant differences emerged between authors' and readers' perceptions (two-tailed t-tests), perhaps suggesting a shared awareness of persuasive strategies. However, significant section-based differences were observed: Abstracts and Introductions were rated as more persuasive than Methods, whereas Discussions were perceived as more persuasive than Results and Methods sections (p < .05). Participants also evaluated whether specific devices (e.g., spin, selective reporting, overgeneralization) constituted attempts to persuade. Six of the nine devices were identified as persuasive by over 80% of respondents (“Yes” or “Sometimes”). These findings, along with the overlap between persuasive devices and questionable research practices, raise questions about their ethical implications for research reliability and transparency.

1124

Reposted by Mark Rubin

Lukas Röseler @aufdroeseler.bsky.social · 29/08/2026

Very interesting simulation by Harper and @nicebread.bsky.social on optimal share of replications. I cannot decide what is most exciting about it so I recommend you read the whole thing.

Plots with tesults from the paper, eg showing how publicatipm boas can diminish (simulated) scientific progress

12911

Reposted by Mark Rubin

Fabian Hutmacher @fabianhutmacher.bsky.social · 25/08/2026

Psychology needs better theories. Sure. But how do we build such theories? And what actually makes a good theory? Together with Alexander Wendt, I have edited a collected volume, "Theory and Model Building in Psychology," which tries to answer some of these questions. doi.org/10.1007/978-...

doi.org

Client Challenge

34014

Reposted by Mark Rubin

some-rss-feeds.bsky.social @some-rss-feeds.bsky.social · 24/08/2026

[some-subscribed-rss] New Post: What if publication bias isn’t that? At least not always?, by Jeremy Fox dynamicecology.wordpress.com/2026/0…

043

Mark Rubin @markrubin.bsky.social · 22/08/2026

"Psychologists often see themselves as in the business of simply producing 'effects'. But what is scientifically at stake if a certain effect exists or does not exist? This is often not made clear." New preprint by Zoltan Dienes: doi.org/10.31234/osf...

Abstract Severe theory testing is greatly facilitated by concisely laying bare before the study is run the essence of the scientific process. (See for example the Study Design Template of Peer Community In Registered Reports.) Testing a theory involves testing a nexus of claims each of which plays an inferential role in deciding whether the evidence counts for or against a substantial theory. Each claim should be explicitly stated in a way that shows what is at stake in testing that claim. The substantial theory is the most general claim that could have evidence count against it - and ideally should be explanatory (having an internal structure that is not free to vary). Psychologists have not been very good at explicitly stating what claim is actually at scientific stake. It should be made clear what theory is being tested by each stated prediction. Predictions should be statistically modelled with a specified test that allows evidence both for or against the prediction. The study should be shown to have a size that could show the theory wrong. How the test will be used to draw conclusions for or against each theory should be specified. This whole structure should be stated so concisely and systematically there is no verbiage to hide behind - the skeleton of the scientific process laid naked so that its integrity can be easily evaluated. The structure can be debated, then the inferential chain nailed down as data collection commences. Researchers can change their mind later - but in full transparency of what is happening.

2182

Reposted by Mark Rubin

John Flournoy @johnflournoy.science · 20/08/2026

This gets a lot of play in UG and early grad methods classes but SO little attention in actual practice. I mean, we think about it, yeah, but not very rigorously.

1154

Reposted by Mark Rubin

Olivier Corneille @ocorneille.bsky.social · 20/08/2026

Recommendations for addressing demand artifacts have been discussed in a fragmented way, and some widely used approaches have notable limitations. In this new article, my co-authors and I provide a structured review of leading methods in the field:

1107

Mark Rubin @markrubin.bsky.social · 20/08/2026

“An intuitively appealing position—widely endorsed by philosophers and scientists—is that we ought to include our total relevant evidence: this is known as the requirement of total evidence (RTE).”

270

Mark Rubin @markrubin.bsky.social · 20/08/2026

New review of methods to address demand characteristics by @ocorneille.bsky.social, Chloé Bernard, and @peterlush.bsky.social doi.org/10.3758/s134...

Demand artifacts stem from participants’ beliefs about treatment effects and motivations to respond in accordance with these beliefs. These artifacts typically build on subtle cues that exert extraneous influences unbeknownst to researchers, through processes that are not part of their stated theory. Demand artifacts threaten the internal and external validity of studies, and they have been supported across a wide range of psychological domains (e.g., clinical, cognitive, and social psychology, psychology of perception) and beyond (e.g., consumer research, experimental economics, psychosomatic medicine, neurosciences). This article reviews prominent methods used to address them. Our discussion is structured around Corneille and Lush's (2023) framework of demand effects, which distinguishes between participants’ hypothesis formation, motivation, and implementation strategies. We conclude this review by discussing three fundamental questions of broad interest: when should we care about demand artifacts, what is the best approach for addressing them, and how can we usefully build on demand artifacts?

12110

Mark Rubin @markrubin.bsky.social · 20/08/2026

“Trustworthy scientific knowledge requires more than replicability. It flourishes when a scientific community is sufficiently diverse (in people and in methods) and open to criticism.”

2164

Mark Rubin @markrubin.bsky.social · 18/08/2026

“Having a 50% replication rate in social and behavioural studies is much better than it has been made to sound. The results from the SCORE project should be considered a reassuring sign of good science prevailing.” New article by @mframos.bsky.social doi.org/10.1093/jrss... #stats #MetaSci

1120

Mark Rubin @markrubin.bsky.social · 14/08/2026

"Some participants also expressed concern that sharing too much information may obscure rather than clarify." #MetaSci #OpenSci

030

Reposted by Mark Rubin

Society for the Improvement of Psychological Science (SIPS) @improvingpsych.org · 13/08/2026

Join us for our first SIPS Pop-Up Event on September 3, 4-5:30 PM CET, an unconference on “Building Better Science Systems | Meta-scientific Assets & Co-Designing a Shared Agenda for Rigorous Research”, led by FORRT director Flávio Azevedo! Register here: forms.gle/cLDS7nfnuwoU... #OpenScience

077

Reposted by Mark Rubin

Russell J. Funk @russellfunk.bsky.social · 12/08/2026

Finally setting up over here. I study innovation and the science of science at the University of Minnesota—how new ideas emerge, spread, and get built on. Looking forward to finding the metascience crowd on this side.

141

Reposted by Mark Rubin

Protzko @protzko.bsky.social · 11/08/2026

What do researchers believe the replication rate of their field is? What do they think is an acceptable %? Lots of debate but not a lot of data; so we asked! 1,826 researchers in 11 fields said they thought ≈56% of studies in their field are replicable, <≈ 60% is a problem. osf.io/preprints/ps...

093

Mark Rubin @markrubin.bsky.social · 10/08/2026

"Were the proposed cuts to be implemented, the entire philosophy of science community in the UK would suffer." #UKHE #AcademicSky #PhilSci #PhilSky #HistSci

098

Mark Rubin @markrubin.bsky.social · 10/08/2026

“Failures, mistakes, misunderstandings, misinterpretations, noise, misalignments, anomalies, and bias are not only phenomena that scientists try to avoid by all means. They lie at the very core of how science works, and how knowledge is produced.”

2143

Reposted by Mark Rubin

mj @malthusjohn.bsky.social · 08/08/2026

"This often implies that what counts as knowledge today will reveal itself tomorrow as an error of the past." This latency keeps #science from acting like an individual. Overturning the status quo consensus is quite difficult, usually more difficult the longer those past errors were believed.

2155

Reposted by Mark Rubin

Aaron Fisher @aaronjfisher.bsky.social · 07/08/2026

Attention early career methodologists and quantitative psychologists: Berkeley Psychology is hiring! This is an assistant teaching professor position with potential for security of employment (equivalent to tenure). Please share widely! Thank you! aprecruit.berkeley.edu/JPF05480

aprecruit.berkeley.edu

Assistant Teaching Professor Quantitative Methods-Psychology

University of California, Berkeley is hiring. Apply now!

14542

Mark Rubin @markrubin.bsky.social · 08/08/2026

Hans-Jörg Rheinberger (2026) considers "unanticipated ignorance" in science. "It is an ignorance that we did not know about, one that we were not able to guess unless experimentation confronted us with it." doi.org/10.1007/978-...

Abstract My paper will explore the function of ignorance and error in the scientific research process. Scientific knowledge, as a particular form among many other forms of knowledge, can even be characterized by its attitude toward ignorance and error. The ground-laying epistemic feature of research consists in gaining knowledge about aspects of the world of which the actual knowledge is either scarce or absent altogether. “Specified ignorance,” as discussed by Robert Merton, is a state of affairs where the black spot of ignorance can be pinpointed. But research is also characterized by a kind of ignorance that we can address as “unspecified,” or “nonanticipated,” that is, by ignoring what one does not know. This often implies that what counts as knowledge today will reveal itself tomorrow as an error of the past. I will especially focus on the latter form of ignorance and present a few historical examples from the life sciences of the twentieth century.

2315

Reposted by Mark Rubin

Jeff Greene @jeffgreene.bsky.social · 07/08/2026

Intriguing argument that the necessary, social aspects of our epistemology not only prevent us from exiting echo chambers, it actually requires that we rely on them, and the testimony of others we trust. doi.org/10.1080/0020...

Screenshot of the title page of an article published in the journal "Inquiry" titled "Social epistemology without a net: learning to live with echo chambers."

24511

Mark Rubin @markrubin.bsky.social · 08/08/2026

Scientific reasoning aims to convince other scientists in an epistemic community using a local ecology of acceptable reasons. Excellent work by @heintz-c.bsky.social and Stefaan Blancke

The unreasonable effectiveness of reasoning in the sciences

Highpoints
• Reasoning evolved for persuading others and for not being wrongly persuaded by
them, rather than for discovering new truths about the world.
• Science succeeds because ordinary reasoning is exercised within a specific social
and cultural setting.
• That setting comprises an ecology of reasons, an arrangement for distributing
evaluation, and an incentive structure.
• Six reasons explain why scientific reasoning is warranted and creative.

1236

Reposted by Mark Rubin

John Drury @profjohndrury.bsky.social · 04/08/2026

I will be advertising a funded PhD studentship shortly, for a project on adding psychological theory (social identity) to modelling of epidemics ( @leverhulme.ac.uk funded, with Marijn Stok). The studentship will start in January 2027. Get in touch now for informal enquiries.

02027

Reposted by Mark Rubin

Laura K. Nelson @lauraknelson.bsky.social · 04/08/2026

Who's up for a good old fashioned (but short, bc I'm tired) paper thread? Thrilled to be in this special issue of Gender & Society (an absolute dream!), where we propose a feminist-inspired extension of how to theorize and model the spread of ideas in science + journals.sagepub.com/doi/10.1177/...

What Gets Lost in Translation? Epistemic Tensions between Translation and Diffusion in Practice-Oriented Scholarship

Abstract
This paper proposes a feminist-informed metascience theory to explain the interconnected processes of idea creation, translation, and diffusion. Drawing on feminist critiques of science centered on perspective, power, and praxis, we develop a creation–diffusion model that jointly examines how ideas are created and then how the content of ideas influences their spread across academic fields. Using a nationally funded gender equity program as a case of a practice-oriented knowledge-production community, we analyze how gender equity research concepts were translated and diffused through the program-funded publications. We operationalize substantive engagement with gender equity research using word embeddings and Concept Mover’s Distance. Then, with additional contextual variables, we measure (1) the predictors of this engagement (ideas) and (2) the impact of engagement on citation counts and citation interdisciplinarity (the diffusion process) across social science and STEM fields. Our findings show that, in addition to context, the content of the idea matters for diffusion: Engagement with critical dimensions of gender equity research, such as feminism and structural concepts, diffused less often, whereas other dimensions, such as gender and methods, achieved broader uptake. Our empirical findings highlight the role of perspectives and epistemic power in shaping both knowledge production and diffusion. Our theoretical framework and methods demonstrate how metascience can incorporate feminist theory directly into diffusion studies, and why it should.

12412

Reposted by Mark Rubin

Berna Devezer @devezer.bsky.social · 03/08/2026

Mark has kindly invited me to do this interview. How I got into metascience, how my path diverged from the mainstream and more if you're interested. I enjoyed thinking about his questions. Hope you enjoy reading!

35821

Mark Rubin @markrubin.bsky.social · 03/08/2026

“Contrary to the common assumption that journals reject null results, we found that most projects halt during early data analysis or presentation stages.” New work by @bartekmacka.bsky.social et al. Open Access: doi.org/10.1007/s112...

The “file drawer problem” – where study results remain unpublished – distorts scientific evidence and wastes resources, yet its impact on experimental philosophy has remained unexplored. We conducted the first dedicated survey of experimental philosophers to investigate the prevalence, causes, and potential solutions to research attrition in this field. Contrary to the common assumption that journals reject null results, we found that most projects halt during early data analysis or presentation stages. The primary drivers for this attrition are resource constraints and competing priorities rather than statistical non-significance or rejection during peer review. Although researchers are highly aware of the file drawer problem, they view “not publishing in a timely manner” as a relatively acceptable practice compared to other questionable research practices. While the adoption of open-science tools like preprints is growing, few researchers utilize micropublications to share result-independent findings. Participants cited the “extra effort” required to share results as a primary barrier. We conclude that the file drawer in experimental philosophy is often populated by “homeless” data that lack the coherent narrative required for traditional publication. To address this, we recommend adopting low-friction dissemination formats, such as standardized templates and brief reports.

42911