It was also my pleasure to contribute to a @ukrepro.bsky.social project that's informed the new UKRI policy: w/colleagues at Bristol, Edinburgh, Glasgow, Leicester, PLoS, & OpenAIRE, we assessed researcher practice against MRC's expectations for research data. Findings here: doi.org/10.5281/zeno...
doi.org
Monitoring Data Availability Statements and FAIR Data Practices in Medical Research Council (MRC) -Funded Research
The FAIR Guiding Principles have become a central framework for research data management and sharing. This study assessed the presence and quality of Data Availability Statements (DASs) against MRC policy expectations and the feasibility of assessing Findability and Accessibility using Data Availability Statements and repository metadata from MRC-funded publications published in 2023. Headlines: DASs were present in 82.3% of sampled publications, but only 58.6% were judged acceptable. Nearly 10% of Digital Object Identifiers (DOIs) did not resolve to a landing page. Repository metadata was generally strong once data had been deposited. Comparison of manual assessments with DataSeer's automated extractions demonstrated variable overall agreement. DataSeer’s extraction identified the presence or absence of a Data Availability Statement with 95.0% accuracy, indicating that DAS detection could be reliably automated at scale. Agreement was lower when assessing whether data had been shared, with DataSeer correctly identifying the status of shared data in 74.8% of cases relative to manual classifications. Performance also varied across specific statement types. For example, DataSeer correctly identified that data had been shared in 91.7% of cases where the manual assessment classified data as being available via repository deposition and correctly identified that data had not been shared in 80.0% of cases where data were available only upon request. OpenAIRE's conservative DAS extraction identified 51.4% of manually verified DASs. Follow-up analysis identified actionable heading variants that have already informed improvements in OpenAIRE’s DAS mining. The identifier-mining results should be interpreted separately: methods designed for full publication text were applied here only to the extracted DAS snippets, so the limited identifier yield does not assess OpenAIRE’s general full-text mining capability. The findings report herein suggest that, with further refinement, automated approaches based on DataSeer’s pipeline could provide an accurate and scalable method for monitoring data-sharing practices, although targeted validation is likely to remain necessary for more nuanced or ambiguous statement types. Whereas the UKRN Open Research Indicator pilots focused on the development and validation of generic indicator specifications, this study examined their applicability within a major biomedical funding portfolio and assessed their potential suitability for routine policy monitoring. The project therefore presents evidence on the practical challenges of implementing indicator-based monitoring in a discipline-specific funder context and informs future dashboard development.