iiinova — where your digital self keeps evolving

Alberto de León

@aldeleon.bsky.social

Post-Doc and Teaching Fellow at @uc3m Previously @strath and @ucd Political Parties | Decentralization | Text-as-data albertodeleon.es #922

Posts Replies Media

Alberto de León @aldeleon.bsky.social · 28/08/2026

A new academic year, a new publication to share! 📚🎉 My article “Asymmetric contagion: the radical right and the erosion of parliamentary civility” is now published in Parliamentary Affairs! Link: academic.oup.com/pa/article-a...

111

Reposted by Alberto de León

Rubén García del Horno @rgarciadelhorno.bsky.social · 05/03/2026

‼️📆 Call for papers – The politics of place: geographic divides in contemporary democracies (AECPA, Granada, 9-11 Septiembre) Con @asanchezgarcia.es @aldeleon.bsky.social @juanperezrajo.bsky.social te invitamos a que nos envíes tu propuesta antes del 31/03 👇 www.aecpa.es/es-es/the-po...

0105

Reposted by Alberto de León

Stefan Müller @stefanmueller.bsky.social · 26/11/2024

New preprint with the fantastic team at @snsf-ch.bsky.social and @ucddublin.bsky.social. Important findings: – Quality of training data more important than quantity – Separate binary classifiers performed best – Fine-tuned transformers outperformed few-shot LLMs 📄 arxiv.org/abs/2411.16662 1/3

Peer review in grant evaluation informs funding decisions, but the contents of peer review reports are rarely analyzed. In this work, we develop a thoroughly tested pipeline to analyze the texts of grant peer review reports using methods from applied Natural Language Processing (NLP) and machine learning. We start by developing twelve categories reflecting content of grant peer review reports that are of interest to research funders. This is followed by multiple human annotators’ iterative annotation of these categories in a novel text corpus of grant peer review reports submitted to the Swiss National Science Foundation. After validating the human annotation, we use the annotated texts to fine-tune pre-trained transformer models to classify these categories at scale, while conducting several robustness and validation checks. Our results show that many categories can be reliably identified by human annotators and machine learning approaches. However, the choice of text classification approach considerably influences the classification performance. We also find a high correspondence between out-of-sample classification performance and human annotators’ perceived difficulty in identifying categories. Our results and publicly available fine-tuned transformer models will allow researchers and research funders and anybody interested in peer review to examine and report on the contents of these reports in a structured manner. Ultimately, we hope our approach can contribute to ensuring the quality and trustworthiness of grant peer review.

Comparison of different classification approaches

1319