iiinova — where your digital self keeps evolving

MetaROR

@metaror.bsky.social

457 followers 129 following 140 posts

MetaROR is a community initiative led jointly by RoRI and AIMOS. It provides a platform that leverages the publish–review-curate model to improve the dissemination and evaluation of metaresearch.

Posts Replies Media

MetaROR @metaror.bsky.social · 05/10/2026

We are very grateful to all of those involved in getting this evaluation out: Authors (@hldejonge.bsky.social, Katharina Rieck) Reviewers (anonymous, @neesjanvaneck.bsky.social) Editors (@stephenpinfield.bsky.social, Kathryn Zeiler)

MetaROR @metaror.bsky.social · 05/10/2026

Both reviewers found the work 'timely' and 'useful', in particular the publisher-level results. They do suggest adding background information, clarifying the methodology as well as more interpretation of the found results.

100

MetaROR @metaror.bsky.social · 05/10/2026

You might be aware that Crossref records can contain funding metadata. But how complete is their coverage? According to this new paper, an approximate 30% of funding acknowledgments do not get deposited in Crossref. 👇 Read the evaluation now on MetaROR: metaror.org/article/lost...

Lost in transition. Quantifying the funding metadata gap in Crossref - MetaROR

MetaROR @metaror.bsky.social · 01/10/2026

👏 We are, as ever, grateful to the author (Heidi Imker), reviewers (Dorothea Strecker, Rob Hooft) and editors (@kathleen-gregory.bsky.social, Kathryn Zeiler) for their contributions.

MetaROR @metaror.bsky.social · 01/10/2026

How sustainable are the databases that life sciences researchers depend on? This new study examined 9 of them to find out. Both reviewers find the study timely and valuable. One reviewer suggests another level of synthesis in the thematic analysis would be helpful. 👉 Read now on MetaROR

Sustainability during instability: Long-lived life science databases and science funding outlook in the United States - MetaROR

MetaROR @metaror.bsky.social · 30/09/2026

The reviewers (@alexrushforth.bsky.social and Thed van Leeuwen) express some reservations on the purpose and intended audience and question the role of Marxist value theory in the framework. 🙏 We are grateful to the authors, reviewers, and editor (@ludowaltman.bsky.social) for their work on this

MetaROR @metaror.bsky.social · 30/09/2026

This article by Yu Zhu and Jiyuan Ye presents a theoretical framework for research evaluation developed in China, highlighting the current binary opposition between qualitative & quantitative methods in evaluation practices. 👉 Read the assessment, reviews, author response & full article on MetaROR

Beyond the binary: The system of all-round evaluation of research and its practices in China - MetaROR

MetaROR @metaror.bsky.social · 22/09/2026

👏 Thanks to the authors (Rolando Villaseñor, Heinrich Nax), reviewers (Marin Dujmović, @mmpittelkow.bsky.social), and editors ( @denolmo.bsky.social, Kathryn Zeiler)

000

MetaROR @metaror.bsky.social · 22/09/2026

This new study takes a meta-perspective on reproducibility by examining replication projects themselves. The reviewers agree the manuscript addresses an important and worthwhile question but identify fundamental problems that currently prevent confidence in the study results. 👉 Read now on MetaROR

A meta-perspective on reproducibility and replication rates - MetaROR

MetaROR @metaror.bsky.social · 08/09/2026

🙏 We thank our author (@dwh.bsky.social), reviewers ( Bianca Kramer, Cameron Neylon, Neil Jacobs) and editors ( @jasonchin.bsky.social, Kathryn Zeiler) for their contribution.

MetaROR @metaror.bsky.social · 08/09/2026

The reviewers credit the work for supplying the debate with vocabulary and analytical tools, yet they share the same criticism: the innovation annalus as described "follows from the construction of the model rather than from evidence about the system it describes".

100

MetaROR @metaror.bsky.social · 08/09/2026

This new MetaROR-evaluated paper introduces "the innovation annulus: the zone between freely available structured data and the advancing frontier of commercially refined knowledge products," which is "a permanent, functional feature of the ecosystem -- not a pathology to eliminate." 👇 Read now

Market dynamics, governance and open research metadata in the AI era - MetaROR

MetaROR @metaror.bsky.social · 07/09/2026

We would like to thank all involved: - Authors: Lee Jones, @aidybarnett.bsky.social, Gunter Hartel, Dimitrios Vagenas - Reviewers: Gabriela Dârzan, @khinsen.net - Editors: Wolfgang Kaltenbrunner, Kathryn Zeiler

MetaROR @metaror.bsky.social · 07/09/2026

Our reviewers praise its methodology, framework and pedagogical value. They also have some remarks, particularly on the conceptual framing of "inferential reproducibility".

MetaROR @metaror.bsky.social · 07/09/2026

Here is the evaluation for "an empirical assessment of the inferential reproducibility of linear regression analyses in health and biomedical research". It demonstrates "substantial violations of key modelling assumptions" that often went unnoticed by authors and reviewers. 👇 Read it on MetaROR

An empirical assessment of inferential reproducibility of linear regression in health and biomedical research papers - MetaROR

MetaROR @metaror.bsky.social · 02/09/2026

👏 Thanks to the efforts of all involved: authors (@simonporter.co.uk, @dwh.bsky.social), reviewers (@dkochetkov.bsky.social, Erjia Yan and anonymous) and editor (@ludowaltman.bsky.social)

000

MetaROR @metaror.bsky.social · 02/09/2026

2️⃣ Here is a revision of one of our first articles - a large-scale data-driven analysis of the use of initials vs. full first names in the author lists of scientific publications. Reviewers were satisfied with the revisions made, though two of them still raised a few minor issues. 👇 Read it now

The rise and fall of the initial era - MetaROR

MetaROR @metaror.bsky.social · 01/09/2026

After two rounds of MetaROR evaluation, this article was just published in Scientometrics! 👏 Congratulations to the authors and everyone else involved.

link.springer.com

Modeling the global citation network using the scalable agent-based simulator for citation analysis with recency-emphasized sampling (SASCA-ReS) - Scientometrics

The global science literature, represented as a network with articles as nodes and citations as edges, is a rich artifact for scientometric studies. The structure of this network depends on its count ...

MetaROR @metaror.bsky.social · 27/08/2026

2️⃣ The revision of this one is now up! Both reviewers are highly satisfied with the (substantial) changes made to the article, and have no further comments. 🙏 Thanks and congratulations to all involved. 👇 Read both versions, including reviews and assessments, here metaror.org/article/rest...

Restructuring scientific papers for human and AI readers - MetaROR

MetaROR @metaror.bsky.social · 25/08/2026

[3 / 3] 🙏 As ever, we are grateful to the author, the reviewers ( @hldejonge.bsky.social and 2 anonymous reviewers) and the editors (@jabyrnesci.bsky.social and Kathryn Zeiler).

MetaROR @metaror.bsky.social · 25/08/2026

[2 / 3] 📝 The three reviewers praised the extensive reference list, rigorous methodology, and interesting results, whilst also providing recommendations, such as clearer recognition of study limitations and more focus on openness.

100

MetaROR @metaror.bsky.social · 25/08/2026

[1 / 3] 🔀 This study by @eumsoohong.bsky.social examines the interoperability between the South Korean national funder database and two bibliographic databases (Web of Science & OpenAlex). 👇 Read the full evaluation now on MetaROR

Assessing funders’ databases against bibliographic sources: A study of interoperability, metadata quality, and coverage using South Korea’s national R&D database - MetaROR

MetaROR @metaror.bsky.social · 18/08/2026

🙏 We are grateful to our authors (Avihay Cohen, Blanka Ivanović, Anastasiia Iarkaeva, Vladislav Nachev, Evgeny Bobrov), reviewers (@kathleen-gregory.bsky.social, Sara Lafia, Dorothea Strecker) and editor (@ludowaltman.bsky.social) for their contributions!

000

MetaROR @metaror.bsky.social · 18/08/2026

♻️ How to detect reuse of data at scale? This article studies reuse of data sets shared by researchers from a biomedical research institution. 📝 The 3 reviewers are generally positive, calling the work 'thorough' & 'exciting' while providing detailed suggestions to improve it. 👇 Read now on MetaROR

Matching data references and institutional output to map the reuse of biomedical research data - MetaROR

MetaROR @metaror.bsky.social · 12/08/2026

🙏 We're grateful to all authors (Andrea Cadeddu, Alessandro Chessa, Vincenzo De Leo, Gianni Fenu, @fosborne.bsky.social, Diego Reforgiato Recupero, @angelosalatino.bsky.social, Luca Secchi), reviewers (@andre-brasil.bsky.social, @ednoyons.bsky.social) and editor (Kathryn Zeiler) [3 / 3]

MetaROR @metaror.bsky.social · 12/08/2026

📝 Both reviewers found the text well-structured and methodologically clear; their concerns relate to clarification, additional discussion, and minor corrections. In particular, they would like a clearer framing of the central terminology. [2 / 3]

100

MetaROR @metaror.bsky.social · 12/08/2026

🌍 Can the polarity of sustainable development goals within news text be detected? That is the question raised by a new article, evaluated on the MetaROR platform. 👇 Read the full evaluation now on MetaROR [1 / 3]

Polarity detection of Sustainable Development Goals in news text - MetaROR

MetaROR @metaror.bsky.social · 11/08/2026

👏 We are grateful to all those involved in making this evaluation happen: the authors (@aidybarnett.bsky.social, Matt Spick), reviewers (@reeserichardson.bsky.social, Dietmar Wolfram, Jiang Li) and editor (@ludowaltman.bsky.social)

MetaROR @metaror.bsky.social · 11/08/2026

📝 Two of the reviewers are broadly supportive of the paper, yet the third one has fundamental criticisms, for instance about distinguishing between article quality, author confidence, and scientific integrity.

100

MetaROR @metaror.bsky.social · 11/08/2026

🔓 "Who chooses open peer review and is it an indicator of article quality?" That's the subject of "an observational study of PLOS journals" evaluated on MetaROR. 👇 Read on MetaROR

Who chooses open peer review and is it an indicator of article quality? An observational study of PLOS journals - MetaROR

MetaROR @metaror.bsky.social · 06/08/2026

2️⃣ This one came back for a second evaluation round! Both reviewers agree comments were well-addressed and commend authors on on a well-written article, yet still express some doubts. 👇 Read the full second (and first!) evaluation now on MetaROR

Citation accuracy, citation noise, and citation bias: A foundation of citation analysis - MetaROR

MetaROR @metaror.bsky.social · 03/08/2026

👏 Our appreciation goes to the authors (Minhyuk Park, Haotian Yi, Tandy Warnow, George Chacko), reviewers (Erjia Yan and anonymous) and editor (@ludowaltman.bsky.social) for their efforts in getting this out.

000

MetaROR @metaror.bsky.social · 03/08/2026

Here is the revision of an article presenting an agent-based model of citation behavior and compares citation networks generated by this model with real-world citation networks. Reviewers feel most of their comments were addressed, though 1 reviewer mentions two more issues. 👇 Read now on MetaROR

Modeling the global citation network using the scalable agent-based simulator for citation analysis with recency-emphasized sampling (SASCA-ReS) - MetaROR

MetaROR @metaror.bsky.social · 30/07/2026

This one has come back from (extensive) revision! The reviewers applaud the changes, yet still highlight some issues; in particular with regard to the title suggesting generic as opposed to specifically biomedical application. 🙏 Thanks again to all those involved! 👇 Read the evaluation on MetaROR

Use of diverse data sources to control which topics emerge in a science map - MetaROR

MetaROR @metaror.bsky.social · 29/07/2026

👏 We are very grateful to (in no specific order): ✒️ Authors: @hughshanahan.bsky.social, @loubezuidenhout.bsky.social ✏️ Reviewers: Jonathan Wheeler, @jessicapolka.bsky.social 📃 Editors: @aidybarnett.bsky.social, Kathryn Zeiler

MetaROR @metaror.bsky.social · 29/07/2026

🌐 How accessible are preprint services and where are they located? That is the question addressed by this article evaluated by MetaROR. The two reviewers are generally positive, though they have some notes on some of the figures and would like to see more statistical tests. 👇 Read it now

The localisation and accessibility of preprint services: implications for Open Science - MetaROR

MetaROR @metaror.bsky.social · 28/07/2026

🙏 A big thank you to the authors (Wolfgang Kaltenbrunner, Andrea Chiarelli, @altibel.bsky.social, @stephenpinfield.bsky.social, @ludowaltman.bsky.social, André Brasil), reviewers (@ginnybarbour.bsky.social, Elena Šimukovič), and editors (@aidybarnett.bsky.social, Kathryn Zeiler) for their efforts.

MetaROR @metaror.bsky.social · 28/07/2026

💭 Why is change in scholarly communication so hard to imagine? This article presents findings from a consultation for cOAlition S proposal Towards Responsible Publishing. Reviewers were generally positive but urged the authors to include more detail and potential solutions. 👇 Read now on MetaROR

Why is change in scholarly communication so hard to imagine? Findings from a stakeholder consultation for the cOAlition S proposal ‘Towards Responsible Publishing’ - MetaROR

MetaROR @metaror.bsky.social · 27/07/2026

👏 Great initiative by our friends at PREreview!

MetaROR @metaror.bsky.social · 23/07/2026

🙏 Our appreciation goes out to: ✒️ Authors: @slewis5920.bsky.social, Alexa M. Tullett ✏️ Reviewers: Jelte M. Wicherts, Robin Brooker, @nagyt.bsky.social 📃 Editor: Kathryn Zeiler [3/3]

000

MetaROR @metaror.bsky.social · 23/07/2026

Our three reviewers agree the topic is timely and relevant to the debates on research integrity and the social dynamics of scientific accountability, but diverge on how close the plan is to being ready for data collection. Particularly the proposed analysis plan raises some concerns. [2/3]

100

MetaROR @metaror.bsky.social · 23/07/2026

This research plan outlines a study investigating how people perceive researchers who commit questionable research practices (investigators) compared to those who report them (inspectors). 👇 Full evaluation - including reviews, editorial assessment & full research plan text - is on MetaROR [1/3]

From misconduct to reform: Understanding perceptions of those who commit & call out questionable research practices - MetaROR

MetaROR @metaror.bsky.social · 22/07/2026

Reviewers consider the article "policy-relevant", "timely", and "well-written", while pushing for clearer explanation of the agent-based model's assumptions. 👏 We thank the author (Alex Hulkes), reviewers (@tomstafford.mastodon.online.ap.brid.gy & anonymous) & editor (@ludowaltman.bsky.social)

MetaROR @metaror.bsky.social · 22/07/2026

What is the relative efficiency of one- and two-stage research funding processes? This article studies that, concluding that two-stage processes “may lead to significant cost savings”. 👇 Read the editorial assessment, reviews, the author's response, and the full article text on the MetaROR website

Insights into the relative efficiency of one- and two-stage research funding processes using an Agent Based Modelling approach - MetaROR

MetaROR @metaror.bsky.social · 05/06/2026

The author of this article is Zhicheng Lin, reviewers are David Resnik, Iratxe Puebla, @soilandreyes.bsky.social & one anonymous reviewer, and editors are @alexh.bsky.social and Kathryn Zeiler. 👏 Thanks to all for your efforts!

MetaROR @metaror.bsky.social · 05/06/2026

This article proposes restructuring scientific papers, with front-loaded narratives for humans as well as machine-readable appendices. Reviewers found the idea worthwhile & timely, while suggesting engagement with previous work & needing an update on the AI processing. 👇 Full assessment on MetaROR

Restructuring scientific papers for human and AI readers - MetaROR

MetaROR @metaror.bsky.social · 04/06/2026

🙏 We thank our partner Lifecycle Journal for making this evaluation possible. Thanks also to the authors (@syeducation.bsky.social, @caroharmstrong.bsky.social, @emilyjchan.bsky.social, Abby Person), reviewers (Alexandra Sarafoglou, Marcel van Assen) & editors (Kathryn Zeiler, @denolmo.bsky.social)

lifecyclejournal.org

Lifecycle Journal | Adding trust to your research, from conception through completion

000

MetaROR @metaror.bsky.social · 04/06/2026

📝 Do preregistration & registered reports restrict exploratory research? This research plan aims to find out. The two reviewers agree the question is important but think the proposal as-is doesn't support its claims yet. 👇 Read the full research plan, editorial assessment & reviews now on MetaROR

Evaluating the claim that preregistration and registered reports restrict exploratory research - MetaROR

MetaROR @metaror.bsky.social · 29/05/2026

A second version of this article has now been evaluated - one of the reviewers of the original deems it "very good" and feels comments were adequately addressed. 🙏 Thanks again for everyone involved in making this happen! 📝 Read both evaluations now on MetaROR: metaror.org/article/the-...

The state of bronze open access in Web of Science - MetaROR