Mark Rubin @markrubin.bsky.social · 16h“Instead of asking researchers to always report as many control variables as possible, it is more effective to report only a few that are considered statistically relevant to the target and explicitly justify them as such.” 110
Mark Rubin @markrubin.bsky.social · 16h“According to the model-based framework, an estimation is unbiased just in case all of the statistically relevant variables have been controlled for. This means what variables are worth controlling will change depending on what the research target is.” 100
Mark Rubin @markrubin.bsky.social · 16h“The benefit of the model-based framework is not about meeting the same standard with less content, but rather documenting and systematizing available and unavailable information in a way that makes assessment possible.” 100
Mark Rubin @markrubin.bsky.social · 16h“Questions of model misspecification and sample balance are intrinsically part of the unknown. In other words, if researchers knew that the model was adequate or that the sample was balanced, they would not have conducted the study in the first place.” 100
Mark Rubin @markrubin.bsky.social · 16hFor more on the distinction between design-based and model-based sample representation, see @kinozhao.bsky.social's excellent work in this area: Zhao (2021) Sample representation in the social sciences. Synthese. doi.org/10.1007/s112... Few quotes follow >> 120
Mark Rubin @markrubin.bsky.social · 18h"In the design-based approach, the inference scheme ... [relies] on the sampling design that determines selection probabilities ascribed to sampling units, while in the model-based approach, the inference scheme does not require a sampling design." 140
Mark Rubin @markrubin.bsky.social · 05/10/2026Following on from Olivia and Sean’s suggestion, here’s a list of 250+ articles in the area of critical metascience…sites.google.comMark Rubin - List of Critical Metascience ArticlesWhat is Critical Metascience? Metascience uses a scientific approach to try to improve science. Critical metascience takes a step back to question some of metascience's common assumptions, approaches,... 094
Reposted by Mark RubinOlivia Guest · Ολίβια Γκεστ @olivia.science · 04/10/2026Inspired by @seanpmackinnon.bsky.social on "curating reading lists for others", here's a thread with some of mine for when people want to collaborate: 1) How do we build theory? 2) What does computationalism imply? 3) How do we use models? 4) What happens when these ideas radiate outwards? 👇🏼metatheory.spacemetatheory.spaceHello! 🍃 metatheory.space is a group that works at the foundations of cognitive science. We are interested in the intersections of psychological and computational neurocognitive theorising with critic... 1268
Reposted by Mark RubinJonny Coates @jacoates.bsky.social · 04/10/2026I'm back on the job market as the mat cover I'm doing is half way through. I'm particularly looking for head of open science or community lead roles. Why should you hire me? 👇 #openscience #job #academiajacoates.co.uk 132
Mark Rubin @markrubin.bsky.social · 04/10/2026"Often the process of hypothesis formation receives short shrift in methods education, but we would argue that it is one of the most critical stages of research design." Open Access: doi.org/10.1177/0959... #Methodology #AcademicSky 0158
Mark Rubin @markrubin.bsky.social · 01/10/2026Yes, I was thinking institutional metascience captures those "practical" aspects. Watney and Gustetic's taxonomy breaks the "institutional" down into “innovation economics,” “R&D management,” and “institutional entrepreneurs,” which might be a more useful split, depending on your focus. 000
Mark Rubin @markrubin.bsky.social · 01/10/2026It's good! I suggested an alternative viewpoint here: markrubin.substack.com/p/contained-... 220
Mark Rubin @markrubin.bsky.social · 29/09/2026You could say there's "poor citation fidelity" for that Johnson et al. (1991) study? (It sounds a bit lile an academic euphemism, but it's a thing!) 111
Mark Rubin @markrubin.bsky.social · 28/09/2026I'm stuck for a term, but looking forward to your paper on this! 010
Mark Rubin @markrubin.bsky.social · 28/09/2026But your case sounds like more than just “spinning” (Bajraktari & Munafò, 2026) or “ignoring” (Lakatos, 1978) an anomaly and more like actively *misrepresenting* it!? So, perhaps neither of the above points actually apply!? 110
Mark Rubin @markrubin.bsky.social · 28/09/2026From this perspective, the question is how successful (progressive) is the resulting research programme? Has the theory changed since the first study, and does it now, successfully predict a whole host of impressive things (Y, Z, A, B, and C!) 210
Mark Rubin @markrubin.bsky.social · 28/09/2026The second is Lakatos’ (1978) point that it’s normal for researchers to sweep anomalies under the rug and push ahead with their research programmes regardless. 120
Mark Rubin @markrubin.bsky.social · 28/09/2026Transparency, Replication Rate Targets, and Direct Replicationsmarkrubin.substack.comTransparency, Replication Rate Targets, and Direct ReplicationsCritical Metascience 030
Mark Rubin @markrubin.bsky.social · 28/09/2026Presumably, you don’t think it’s a false positive when you’re disseminating it? 110
Mark Rubin @markrubin.bsky.social · 27/09/2026New "Research Handbook on the Replication Crisis" Edited by David Trafimow with 17 chapters covering issues such as results-blind publishing, preregistration, theory development, and statistical inference. www.e-elgar.com/shop/gbp/res... 083
Mark Rubin @markrubin.bsky.social · 27/09/2026Was that me or someone else? Don't remember anything like that? But this table from Lewandowsky and Bishop (2016) is useful for evaluating these sorts of requests: doi.org/10.1038/5294... 030
Mark Rubin @markrubin.bsky.social · 12/09/2026Chapter by Karen Trew and Denny Benson in the new book "Changing European Identities: Social Psychological Analyses of Social Change" #SocialPsyc #AcademicSkytaylorfrancis.comChanging European Identities | Social Psychological Analyses of SocialIn the 1990s, moves towards federalism within the European Union, and the political transition since the collapse of Communism had given rise to a number of 050
Mark Rubin @markrubin.bsky.social · 12/09/2026"It is, therefore possible that in the light of the continuing violence in Northern Ireland both Protestants and Catholics may find that they can acknowledge their common heritage without threatening their strong group allegiances."taylorfrancis.comDimensions of Social Identity in Northern Ireland | 10 | Changing EuroAcademic and official commentaries generally use the terms “Protestants” and “Catholics” to refer to the two communities in Northern Ireland. However, Whyte 132
Mark Rubin @markrubin.bsky.social · 12/09/2026Chapter in the new Routledge Handbook of Values and Science #MetaSci #PhilScitaylorfrancis.comThe Routledge Handbook of Values and Science | Kevin C. Elliott, Ted RThis is the first-ever handbook to cover the vibrant philosophical literature on values and science. Its 45 chapters—appearing in print here for the first 082
Mark Rubin @markrubin.bsky.social · 12/09/2026Stephen John (2026) considers the complexities of "Transparency in Science." www.taylorfrancis.com/chapters/oa-... 371
Mark Rubin @markrubin.bsky.social · 12/09/2026“Centralized metascience that issues science-wide mandates” is not what science needs, she (@devezer.bsky.social) wrote on the social-media platform Bluesky. “The interventionist, top-down, efficiency-driven approach to science policy depresses me.” 140
Mark Rubin @markrubin.bsky.social · 12/09/2026"Staff members at the agency, who spoke on the condition of anonymity out of fear of retaliation, are concerned that the changes will lead to further cuts for work that has long been core to the agency’s mission: discovery-focused research, led by the curiosity of scientists." #AcademicSky #MetaScinature.comNSF moves to overhaul funding approach in line with White House prioritiesThe major funder of basic science has not said how its new initiatives will be funded — but staff worry streamlined approach will further squeeze cash for core sciences. 198
Reposted by Mark Rubinλ³🎲 @cubiclogic.bsky.social · 11/09/2026Βrexit dividends The north will decline further without international students bsky.app/profile/mark... 171
Reposted by Mark RubinJoe Bak-Coleman @jbakcoleman.bsky.social · 11/09/2026I stand firmly by my assertion that metascientists should have seen this capture of science coming, but there is no reasonable way they can reasonably delude themselves into thinking the new nsf announcements are anything other than epistemic capture. www.science.org/content/arti...science.orgScience’s reform movement should have seen Trump’s call for ‘gold standard science’ coming, critics sayEfforts to improve the rigor of research may have unwittingly handed the administration a way to attack science 29038
Mark Rubin @markrubin.bsky.social · 10/09/2026"The University of Hull has launched a call for volunteers to leave, as a steep decline in international student numbers impacts its finances." #UKHE #AcademicSkyhulldailymail.co.ukUniversity of Hull launches 'voluntary exit' call amid cost-cutting measuresNew Vice Chancellor Tom Lawson also warns 'It is unlikely that this scheme is going to be the only measure that we need to implement' 062
Mark Rubin @markrubin.bsky.social · 09/09/2026Rogue Scholar has now made its >50K blog posts available as PDF files.rogue-scholar.orgRogue Scholar has archived all blog posts as PDFThe Rogue Scholar science blog archive has now archived all its currently 55,730 blog posts as PDF files. These PDF files use the archival PDF/A-3A format and include the full-text HTML blog post with... 033
Mark Rubin @markrubin.bsky.social · 08/09/2026"By stripping away many of the elements of Popper’s philosophy of critical rationalism that make his falsificationist theory of science a rich account of scientific inquiry, the authors are left with an impoverished and rather unsophisticated account of the hypothetico-deductive method." #PhilSci 142
Mark Rubin @markrubin.bsky.social · 08/09/2026Yes, it's a weird term, but it's being used consistent with Knorr-Cetina (1999) www.jstor.org/stable/j.ctv... To me, it sounds more like method/stats based knowledge. 020
Mark Rubin @markrubin.bsky.social · 07/09/2026Yes, the authors make it clear that: "Our results concern only the bias of published estimates. They say nothing about the other pernicious effects of p-hacking, such as inflating the number of false positives, eroding trust in science, or distorting the incentives of career researchers." 110
Mark Rubin @markrubin.bsky.social · 07/09/2026Sorry - here's the link: arxiv.org/abs/2609.05372arxiv.orgDoes p-Hacking Mitigate or Exacerbate the Effects of Publication Bias?This paper studies the effects of p-hacking on the bias of published estimates when papers with statistically significant results are selectively published. We show that fast p-hacking---actions that ... 000
Mark Rubin @markrubin.bsky.social · 07/09/2026Negative Knowledge: "Knowledge about how to get knowledge" “Whilst the contributions of statistical and methodological experts were widely valued, the pursuit of negative knowledge was deeply unappealing to some interviewees." New work by @tayacollyer.bsky.social doi.org/10.1016/j.so... 141
Mark Rubin @markrubin.bsky.social · 07/09/2026“Our results show that it is important to understand p-hacking in the context of the selection environment that produced it. That p-hacking could mitigate the effects of publication bias under extreme selection also hints at a self-correcting tendency in the scientific process.” 252
Mark Rubin @markrubin.bsky.social · 05/09/2026"This study examined the determinants of public trust in scientific findings and found that the communication of OS [open science] practices in layperson texts plays no substantial role in shaping trust." Open Access: doi.org/10.1177/1075... 076
Reposted by Mark Rubinlabfolio.bsky.social @labfolio.bsky.social · 03/09/2026How does the metascience community think about the Contribution Roles Taxonomy (CRediT), in use at various prominent publishers? Is it any good? Does it miss any big categories of work? www.elsevier.com/researcher/a...credit.niso.orgCRediTContributor Role Taxonomy 011
Mark Rubin @markrubin.bsky.social · 02/09/2026The Journal of Trial and Error is looking for a new Editor with an interest in meta-research, open science, or academic publishing. More info: www.linkedin.com/posts/were-l... 096
Mark Rubin @markrubin.bsky.social · 02/09/2026“There is no published study comparing time-to-publication for registered reports against standard articles." "If anyone is interested in trying to collect this data more systematically for political science, please let me @aaronerlich.bsky.social know.” 253
Mark Rubin @markrubin.bsky.social · 02/09/2026#PhilScidoi.orgToward a Philosophy of Error in ScienceAbstract. Philosophers, historians, sociologists, and scientists disagree about the nature of error in science. Some focus on the misframed phenomena and t 040
Mark Rubin @markrubin.bsky.social · 02/09/2026“To err is science.” Stephan Guttinger reviews Douglas Allchin's new book: "Toward a Philosophy of Error in Science" doi.org/10.1093/bios... 1215
Mark Rubin @markrubin.bsky.social · 01/09/2026"What is clear is that more research is needed to determine when, if ever, MC [multiplicity control] is warranted and under what circumstances. In the current systematic review, we are interested in how empirical researchers utilize MC within their research." #stats 050
Reposted by Mark RubinBerna Devezer @devezer.bsky.social · 01/09/2026Been reflecting some more on how for a decade+ there's been a whole outcry about how replications weren't published. Part of it is based on misleading data as it's now made clear (see the thread I'm replying to). But another part was that standalone replication studies weren't being published. 2243