Olmo van den Akker @denolmo.bsky.social · 05/06/2026And some words of warning by @jamiecummins.bsky.social 041
Olmo van den Akker @denolmo.bsky.social · 05/06/2026Some examples by Dvija Mehta of how AI can shake up the foundations of science 110
Olmo van den Akker @denolmo.bsky.social · 05/06/2026Big news at the META/e conference: metacheck, a promising tool for automatically extracting information from scientific manuscripts is beta-launched today. Has the potential to massively increase the efficiency of meta-science projects. www.scienceverse.org/metacheck/scienceverse.orgCheck Research Outputs for Best PracticesA modular, extendable system for checking research outputs for best practices using text search, R code, and/or (optional) LLM queries. 0136
Reposted by Olmo van den AkkerLisa DeBruine @debruine.bsky.social · 05/06/2026@crist14n.bsky.social presented some of the capabilities of metacheck, which has its first beta release today! See an example report that demonstrates all of the current modules. www.scienceverse.org/metacheck/re... 103
Reposted by Olmo van den AkkerDaniel Leising @leising.bsky.social · 06/05/2026Our paper "An analysis of functional relationships between systemic conditions and unethical behavior in German academia" has just been published in Meta-Psychology: open.lnu.se/index.php/me...open.lnu.se LnuOpen | Meta-Psychology 1143
Reposted by Olmo van den AkkerEuropean MS Platform @emspofficial.bsky.social · 24/04/2026#EMSP2026 “Patient registries must reflect patients’ preferences — it all comes down to trust.” A powerful takeaway from @denolmo.bsky.social at the #EMSP2026 Conference. Watch the livestream- eu-admin.eventscloud.com/website/1909... 012
Reposted by Olmo van den AkkerKatie Corker @katiecorker.bsky.social · 07/04/2026The governing body for 14 Dutch universities (UNL) has published a "Vision on Publication Culture" that is so inspiring and forward thinking. Worth a read for those trying to changes publishing and research assessment practices. Take a read: www.universiteitenvannederland.nl/files/public... 43928
Olmo van den Akker @denolmo.bsky.social · 26/03/2026Come join this project to study a form of peer review that focuses on the methods instead of the results. I'm already a postdoc on the team so hit me up if you have any questions! 011
Reposted by Olmo van den AkkerAccountability in Research: Ethics, Integrity, and Policy @accountabilityair.bsky.social · 09/12/2025📝 Latest Article Alert! 📝 "Rethinking (un)blinding in biomedical proposal peer review: A multi-stakeholder qualitative study" Seba Qussini, Farizah Mezer Anami & Kris Dierickx → doi.org/10.1080/0898... #PeerReview #AcademicPublishing #MedicalScience #BioMedicinedoi.orgRethinking (un)blinding in biomedical proposal peer review: A multi-stakeholder qualitative studyMany peer review attributes are widely criticized and poorly investigated, particularly in the context of proposals’ peer review. This study aims to explore stakeholders’ perspectives on the role o... 022
Olmo van den Akker @denolmo.bsky.social · 25/03/2026Should we use error checking tools like Statcheck and GRIM to automatically check for errors in studies in meta-analyses? I wrote a blog post with some insights from a hackathon where we scrutinized this idea. metaresearch.nl/blog/2026/3/...metaresearch.nlChecking for Statistical Errors in Primary Studies: A Way to Improve Meta-Analytic Evidence? — Meta-Research CenterThis blogpost was written by dr. Olmo van den Akker. He is a postdoc in our department working on two projects: a systematic review of studies that simulate p-hacking, and a randomized controlled tria... 0208
Reposted by Olmo van den AkkerPlatform for Young Meta-Scientists (PYMS) @youngmetascience.bsky.social · 24/03/2026📢 New from PYMS! We're launching online lunch meetings. A chance to present your work to other metascience ECRs Share your work in any form: finished projects, preliminary results, a trial-run conference presentation, or ideas you'd love feedback on Want to present? Sign up using the form!forms.glePYMS - Lunch PresentationsBetween 20 and 24 April, PYMS will be hosting a series of online lunch meetings. During these sessions, you can present one of your completed research projects, an ongoing study, or early‑stage ideas ... 0610
Olmo van den Akker @denolmo.bsky.social · 10/03/2026On 24 April, I will speak at the @emspofficial.bsky.social conference about making studies using patient registry data more transparent. It means a lot to me because a close friend was diagnosed with Multiple Sclerosis just last year. Hope to learn a lot there! #EMSP2026 040
Reposted by Olmo van den AkkerSajedeh Rasti @sajedehra.bsky.social · 06/02/2026You still have time to sign up for the upcoming workshop of PMGS. @denolmo.bsky.social will guide you through evaluating and writing high quality preregistration. See more and sign up here: paulmeehlschool.github.io/workshops/pr...paulmeehlschool.github.ioPreregistration in Practice | Paul Meehl Graduate SchoolFebruary 19, 2026 034
Olmo van den Akker @denolmo.bsky.social · 05/02/2026That sounds like a lot! What were the reasons for concerns/retractions? 000
Reposted by Olmo van den AkkerMichèle Nuijten @michelenuijten.bsky.social · 03/02/2026I wrote a blog for the Meta-Research Center expressing my infinite frustration about not getting data. What else is new, you might think? Well, I added an extra layer of annoyance directed at the journals who do NOTHING to enforce promised data sharing. metaresearch.nl/blog/2026/2/...metaresearch.nlPromised Data Unavailable? – I’m Sorry, Ma’am, There’s Nothing We Can Do — Meta-Research CenterThis blogpost has been written by Michèle Nuijten. Michèle is an assistant professor of our research group who investigates reproducibility and replicability in psychology. Also, she is the developer ... 75937
Reposted by Olmo van den AkkerCenter for Open Science @cos.io · 29/01/2026Now available on the OSF as part of a growing collection of preregistration resources, the new EEG & ERP Methods template guides researchers through every stage of ERP study planning. In our Q&A, two of its creators share how the template can help researchers at all stages:cos.ioIntroducing the EEG and ERP Methods Template: Q&A with Gisela Govaart and Antonio SchettinoInterview with Gisela Govaart and Antonio Schettino, developers of the new ERP preregistration template on the Open Science Framework (OSF). 01610
Reposted by Olmo van den AkkerAusHSI @aushsi.bsky.social · 30/01/2026“If fabricated studies make their way into the evidence base, they can mislead real scientists and ultimately slow progress for patients." Published in The BMJ, a new AI tool developed by @aidybarnett.bsky.social and colleagues has exposed the scale of fake studies flooding cancer research.qut.edu.auNew tool exposes scale of fake research flooding cancer scienceA new machine learning tool has identified more than 250,000 cancer research papers that may have been produced by so-called “paper mills”. 03119
Reposted by Olmo van den AkkerPeer Review Congress #PRC11 Coming 2029! @peerreviewcongress.bsky.social · 02/01/2026#OA articles are cited more, help citizen scientists learn more about the topic they've helped on, but there’s little strong evidence they have other long-lasting effects on research, or many economic and social benefits www.science.org/content/arti... @jeffreybrainard.bsky.social @science.orgscience.orgIs ‘open science’ delivering benefits? Major study finds proof is sparseIt’s hard to measure social and economic impacts of making papers and data free, researchers say 044
Olmo van den Akker @denolmo.bsky.social · 26/11/2025Ha, I'm also in Sydney atm! It was pretty fiery when I went for a run this afternoon. Are you here for a conference? 110
Olmo van den Akker @denolmo.bsky.social · 20/11/2025Proposal to use more nicknames when talking about scientific researchers, the fun of which is nicely illustrated by James "cheaters' bane" Heathers in his acknowledgement slide. #AIMOS2025 @jamesheathers.bsky.social 050
Olmo van den Akker @denolmo.bsky.social · 18/11/2025Slide by Lisa Bero on commercial funding of research. No further comment necessary, I think. #AIMOS2025 0112
Olmo van den Akker @denolmo.bsky.social · 18/11/2025What is the best reporting guideline acronym? ROB-ME by @mjpages.bsky.social gets my preliminary vote. Any other good ones, #AIMOS2025?doi.orgROB-ME: a tool for assessing risk of bias due to missing evidence in systematic reviews with meta-analysisVarious methods are available to help users assess whether selective non-publication of studies or selective non-reporting of study results has occurred, but not its impact on a meta-analysis. This li... 041
Olmo van den Akker @denolmo.bsky.social · 01/10/2025Hi GRIOS, is the contact form on your website functional? I sent a message just after Metascience25 but haven't heard back yet. 100
Reposted by Olmo van den AkkerResearch on Research Institute (RoRI) @rorinstitute.bsky.social · 09/09/2025🔓 It's great to see authors sharing their experiences with publishing on MetaROR (MetaResearch Open Review) — our open review platform for metascience using the publish–review–curate model: www.openscience.nl/en/cases/the... 043
Olmo van den Akker @denolmo.bsky.social · 08/09/2025Here's another conference that aims to bridge fields: errorsin.science/pse8/ In Leiden from 11-13 Feb 2026 (submission deadline 15 October)errorsin.sciencePerspective on Scientific Error – 8th Perspectives on Scientific Error Workshop 010
Reposted by Olmo van den AkkerBrian Nosek @briannosek.bsky.social · 04/09/2025We are about a month away from releasing a complete refresh of the OSF user interface. The team has been working on this for a very long time, and we are very excited to be able to share it soon. A preview picture: 914830
Olmo van den Akker @denolmo.bsky.social · 04/09/2025- Journals should state what their aims and scope are from the outset and implement mechanisms to assess whether they achieve those aims. This could also be things like "we want to publish high risk research" - Meta-research is necessary to find out which journals deserve prestige #PRC10 020
Olmo van den Akker @denolmo.bsky.social · 04/09/2025- "The replication crisis forced changes in transparency for the research itself, but not for the publication process" - We need to raise our expectations for journals? How? Nullius in verba (don't take their word for it!) #PRC10 120
Olmo van den Akker @denolmo.bsky.social · 04/09/2025Many interesting tidbits in @simine.com's talk. A selection: - Journal prestige depends on factors like aims and scope, selectivity, and impact factor, but changes in these factors do not always lead to changes in journal prestige - journal prestige is sticky #PRC10 120
Olmo van den Akker @denolmo.bsky.social · 04/09/2025There is also a publish-review-curate publishing platform specifically dedicated to meta-research: metaror.org Send your studies on peer review there and be part of the future of science! (CoI statement: I'm an ERC representative at MetaROR) #PRC10 063
Olmo van den Akker @denolmo.bsky.social · 04/09/2025eLife (talk by Nicola Adamson) uses a publish-review-curate method and uses common terms to assess manuscripts. For strength of evidence: exceptional, compelling, convincing, solid, incomplete, & inadequate For significance of findings: landmark, fundamental, important, valuable, & useful #PRC10 251
Olmo van den Akker @denolmo.bsky.social · 04/09/2025New peer review dataset incoming! Involves authors, topic area, editorial decision, author characteristics (institutional prestige, region, gender), BoRE evaluations, review characteristics (length, sentiment, z-score, reviewer gender). (Talk by Aaron Clauset) #PRC10 040
Olmo van den Akker @denolmo.bsky.social · 04/09/2025Christos Kotanidis checked differences in abstracts between submissions and published papers & assessed whether these differences indicated higher or lower research quality. Abstracts typically improved, especially in big five medical journals. Evidence for the effectiveness of peer review? #PRC10 010
Olmo van den Akker @denolmo.bsky.social · 04/09/2025Andrea Corvillon on distributed vs. panel peer review at the ALMA Observatory: Most experienced PIs no longer have the best ranks in a distributed review system, but why that is remains unclear. #PRC10 010
Olmo van den Akker @denolmo.bsky.social · 04/09/2025Interesting to see that the conference review process (and publishing norms) are do different in the field of computer science compared to other fields. How do these differences come about? Fundamental differences between fields or chance and inertia? #PRC10 110
Olmo van den Akker @denolmo.bsky.social · 04/09/2025Alexander Goldberg did it by a 7-point Likert scale for overall review quality but also by assessing 4 sub-categories: reviewers' understanding of the paper, whether important elements were covered, whether reviewers substantiated their comments, and the constructiveness of reviewer comments. 000
Olmo van den Akker @denolmo.bsky.social · 04/09/2025Di Girolamo explains why the use of the phrase "to our knowledge" lacks reproducibility and accountability. Good trigger to make an edit in a grant proposal I'm writing. #PRC10 030
Olmo van den Akker @denolmo.bsky.social · 04/09/2025Note by Yulin Yu: Data repurposing may serve as an essential driving mechanism driving scientific innovation BUT may not always garner immediate recognition. 000
Olmo van den Akker @denolmo.bsky.social · 04/09/2025Data repurposing: taking existing data and reusing it for a different purpose. (Presentation by Yulin Yu) Studies repurposing data are at higher risk of bias, so make sure to preregister them (check here for a template): research.tilburguniversity.edu/en/publicati... #PRC10 191
Olmo van den Akker @denolmo.bsky.social · 04/09/2025Different findings in terms of time and industry funding than in an earlier meta-analysis by Robert Thibault and others: www.medrxiv.org/content/10.1... Can this discrepancy be explained by the use of AI? #PRC10medrxiv.org 010
Olmo van den Akker @denolmo.bsky.social · 04/09/2025Ian Bulovic used OpenAI's GPT to assess selective outcome reporting. Findings: - Much outcome switching but decrease over time - Industry-sponsored trials most at risk - Assessing outcome switching may seem trivial but is even hard for human coders #PRC10 100
Olmo van den Akker @denolmo.bsky.social · 04/09/2025Ioannidis: Do we enough evidence for your proposed actions to improve peer review? Macleod: The evidence is thin, partly because many journals are hesitant to accommodate meta-research, like RCTs. #PRC10 041
Olmo van den Akker @denolmo.bsky.social · 04/09/2025A meta-perspective by Malcolm Macleod on the presentations at #PRC10. Are we going for low hanging fruit too much in research on peer review / publication? 143
Olmo van den Akker @denolmo.bsky.social · 04/09/2025A question to kickstart day 2 of #PRC10: How would you measure the quality of peer reviews in a scientific study? Single question? Scale? How many raters? AI? 120
Olmo van den Akker @denolmo.bsky.social · 03/09/2025Leslie McIntosh: Markers of (dis)trust in science: Pay attention to email addresses (use of hotmail.com and underscores) and institutional affiliations (new and unknown organizations without verifiable addresses) #PRC10 161
Olmo van den Akker @denolmo.bsky.social · 03/09/2025See also this piece by Tony Ross-Hellauer and Serge Horbach: doi.org/10.1371/jour...doi.orgOpen peer review urgently requires evidence: A call to actionOpen Peer Review is gaining prominence in attention and use, but to responsibly open up peer review there is an urgent need for additional evidence. This Perspective proposes a preliminary research ag... 000
Olmo van den Akker @denolmo.bsky.social · 03/09/2025John Ioannidis: "We need more RCTs" I agree, so here is an urgent call to the representatives of journals at #PRC10: Let's empirically test suggested improvements to peer review like open reports, open identities, structured review, results-free review, collaborative review, etc. Get in touch! 4144