Sign in

Jamie Cummins

@jamiecummins.bsky.social
3.1K followers 816 following 1.1K posts

Advanced postdoc at University of Bern. Also sometimes @bennettoxford.bsky.social. Metascience stuff. Creator and developer of RegCheck (regcheck.app). STAR Editor at Psych Science. @error.reviews 🇮🇪

PostsRepliesMedia
Reposted by Jamie Cummins
Richard Van Noorden @richvn.bsky.social · 02/10/2026
NEW: researchers say one 'zombie' study in Iran, 20 years ago, has skewed the world's medical evidence base for using Prozac to treat depression in children. They stop short of saying don't use Prozac this way, but want clinical guidelines revised. @nature.com www.nature.com/articles/d41...
nature.com
Prozac use for childhood depression skewed by single flawed medical trial
One small study in Iran swayed meta-analyses in favour of antidepressant use in children, researchers say.
1218
Reposted by Jamie Cummins
Bennett Institute for Applied Data Science @bennettoxford.bsky.social · 02/10/2026
📣Cory Doctorow will deliver the closing keynote at the 2026 Bennett Oxford Symposium! Open to the public. Already attending? No need to register again. 🔗 www.eventbrite.co.uk/e/cory-doctoro… 📸 Anna Olthoff
052
Reposted by Jamie Cummins
Henry Carnell @hencarnell.bsky.social · 01/10/2026
New research: “AI-generated survey results consistently differ from human surveys“
pewresearch.org
How well synthetic samples replicate public opinion
Synthetic survey respondents tend to exaggerate differences between demographic subgroups and portray some responses as nearly universal.
12512
Reposted by Jamie Cummins
Dirk Wulff @dirkwulff.bsky.social · 01/10/2026
📣 Delighted to share that I have joined WU (Vienna University of Economics and Business) as Professor of Artificial Intelligence and Behavioral Science. Announcements for at least one postdoc-level position will follow soon. Please reach out if you are interested.
67916
Reposted by Jamie Cummins
Ian Hussey @ianhussey.mmmdata.io · 29/09/2026
Hayes has been banging this drum about idionomic analyses for 6+ years, with folk like @jamiecummins.bsky.social pointing out that its reinventing-the-wheel-but-worse for just as long. My experience is that the ACT community is insufficiently interested and aware of existing methods+stats work.
12312
Jamie Cummins @jamiecummins.bsky.social · 28/09/2026
My lukewarm take is that when people disagree about whether LLMs can do X (be it reasoning, creativity, etc), the substantive disagreement is really about what X means, rather than about LLMs themselves. Aka Wittgenstein was right
1173
Reposted by Jamie Cummins
Magnus Johansson @pgmj.bsky.social · 28/09/2026
Our commentary "A Revolution in Search of an Estimand: On the Idionomic Proposal of Hayes et al. (2026)" is now available as a preprint: osf.io/preprints/ps... With @rpsychologist.com, @matti.vuorre.com, and Nils Hentati Isacsson.
osf.io
OSF
03118
Reposted by Jamie Cummins
Institute for Replication @i4replication.bsky.social · 25/09/2026
Over the past two years, I4R partnered with Psychological Science (@psychscience.bsky.social) on a large reproducibility project. 67 independent teams reproduced 64 articles published in 2024–2025, about 44% of eligible papers. Paper: www.econstor.eu/bitstream/10... Here's what we learned 🧵
18958
Reposted by Jamie Cummins
Saloni @scientificdiscovery.dev · 25/09/2026
Happy Friday! Iʼve created a game for you to play! Will AI cure all diseases in 10 years? Inspired by bottlenecks discourse, I created a game to help build your intuitions. Play it here: game.clinicaltrialsabundance.blog Naturally, it’s bottle-themed with bubbles and there are leaderboards too.
33810
Jamie Cummins @jamiecummins.bsky.social · 24/09/2026
I sincerely hope this paper will be seen as a turning point in how the medical sciences think and approach the issue of trustworthiness assessment (and I hope the social sciences follow suit!). Congrats to @jdwilko.bsky.social and colleagues (incl. @ianhussey.mmmdata.io and @lhdjung.bsky.social)!
12210
Reposted by Jamie Cummins
The Cochrane Collaboration @cochrane.org · 24/09/2026
INSPECT-SR – a tool for assessing the trustworthiness of randomized controlled trials – has been published today in @bmj.com, giving researchers a structured way to identify problematic studies before they're included in systematic reviews. 🔗 www.cochrane.org/about-us/new... #ResearchIntegrity
“Systematic reviews are only as credible as the studies they include.” — phrased as a Cochrane senior research integrity editor; a magnifying glass rests on stacked papers in the background to emphasize careful scrutiny of research.
03124
Reposted by Jamie Cummins
Randy Ellis @randalljellis.bsky.social · 23/09/2026
Here’s my conversation with Chris Chambers on Metascience Matters. Chris was the first journal editor to offer registered reports, starting in 2013 at Cortex. youtu.be/PgZjgZlxsYw?...
youtu.be
Registered reports | Chris Chambers
YouTube video by Metascience Matters
11912
Reposted by Jamie Cummins
Alexander Wuttke @kunkakom.bsky.social · 23/09/2026
We are hiring at LMU Munich: two PhD positions and one postdoc on democracy, political psychology and computational social science. Start: January 2027 or as soon as possible. We review applications on a rolling basis until October 20. Details below. Reposts very welcome! 1/7
25981
Jamie Cummins @jamiecummins.bsky.social · 23/09/2026
We just published a new paper (led by Love von Schreeb) demonstrating how large language models can be used to link published results to trial registrations with very high accuracy! Now out in Journal of Clinical Epidemiology - check it out: www.jclinepi.com/article/S089...
jclinepi.com
TrialScout links published results to trial registrations using a large language model
Multiple stakeholders need to locate results of registered clinical trials but frequently struggle to find them. Summary results of clinical trials are often not published in trial registries, and pub...
1187
Reposted by Jamie Cummins
Eileen Clancy @eileenclancy.bsky.social · 16/09/2026
RegCheck leverages LLMs and AI to automatically compare preregistration plans with scientific publications. This lets researchers effortlessly identify if and how executed studies deviated from the initial plan regcheck.app
regcheck.app
RegCheck
RegCheck is an AI tool to compare preregistrations with papers instantly.
021
Reposted by Jamie Cummins
Rafael M Batista @rafmbatista.bsky.social · 13/09/2026
I’ve got a @semble.so Collection of #AI tools I’ve come across specifically for academic research. Most of these Ive tried and found them user-friendly-ish. If you use something not on the list, please share it :) semble.so/profile/rafm...
semble.so
AI Tools for Academic Research (by Rafael M Batista) — Semble
View Rafael M Batista's collection on Semble
195
Reposted by Jamie Cummins
Sandra Grinschgl, (she/her) @grinschglsandra.bsky.social · 13/09/2026
Happy Sunday morning! This article was finally retracted: psycnet.apa.org/fulltext/202... Unfortunately the author continues as is - no accountability. Curious if we will see more sloppy work from her published in Psych journals soon-ish 🧐
psycnet.apa.org
APA PsycNet
094
Reposted by Jamie Cummins
Luna Fazio @lunafazio.bsky.social · 09/09/2026
Had a Zoom meeting earlier today where a lot of great exchanges were happening in the chat, so we wanted to save the log for later. Zoom exports a plain text file, so I made a tool to convert that into something that's more convenient to interact with. Sharing in case anyone else finds it useful!
Example of what a zoom chat log looks like (fictional conversation)The chat log after being converted into an interactable html
153
Reposted by Jamie Cummins
Ruben C. Arslan @ruben.the100.ci · 11/09/2026
An interesting thing is happening to me at the moment. I have a standing bug bounty policy for my first authored scientific work. That means, I've committed to pay people who report bugs in my code, up to 750€ for severe errors. Since GPT-6 Astra came out, I'm now getting several bug reports daily.
39334
Jamie Cummins @jamiecummins.bsky.social · 10/09/2026
Machine learning and generative LLMs are subject to many of the same drawbacks and issues as general empirical research in the typical publishing system. In this piece now published at NHB we argue that, like for empirical research, ML and LLMs can benefit from the use of Registered Reports.
02210
Reposted by Jamie Cummins
Nature Human Behaviour @nathumbehav.nature.com · 10/09/2026
How Registered Reports can improve trust in machine learning research
dlvr.it
How Registered Reports can improve trust in machine learning research
Nature Human Behaviour, Published online: 10 September 2026; doi:10.1038/s41562-026-02586-2How Registered Reports can improve trust in machine learning research
084
Reposted by Jamie Cummins
Kristin Jankowsky @krijanko.bsky.social · 10/09/2026
Out now, with @jamiecummins.bsky.social & @uschroeders.bsky.social: why ML research benefits from Registered Reports, and where the format reaches its limits. Table 1 goes through 7 scenarios, from confirmatory prediction to climbing a benchmark leaderboard. Free view-only link: rdcu.be/MCamygrKHKxa
rdcu.be
042
Reposted by Jamie Cummins
Journal of Clinical Epidemiology @jclinepi.bsky.social · 07/09/2026
TrialScout links published results to trial registrations using a large language model - Journal of Clinical Epidemiology www.jclinepi.com/article/S089...
jclinepi.com
TrialScout links published results to trial registrations using a large language model
Multiple stakeholders need to locate results of registered clinical trials but frequently struggle to find them. Summary results of clinical trials are often not published in trial registries, and pub...
153
Reposted by Jamie Cummins
Institute for Replication @i4replication.bsky.social · 08/09/2026
1/ What is the prevalence of personally identifiable information (PII) in author-collected data publicly shared in the social sciences? We audited 327 replication packages from 11 leading journals in economics, political science, and psychology Answer: surprisingly high 🧵
25537
Reposted by Jamie Cummins
Joachim Baumann @joachimbaumann.bsky.social · 04/09/2026
After Science News, our recent ICML paper also got featured in @science.org! Article: www.science.org/content/arti... Paper: arxiv.org/pdf/2605.03202 @dirkhovy.bsky.social @sanmikoyejo.bsky.social #science #PeerReview #research
Screenshot of an article in Science Magazine on the opportunities and risks of AI used for peer review featuring an ICML conference paper by Joachim Baumann et al., 2026.
1104
Reposted by Jamie Cummins
Sabrina Norwood @sabrinanorwood.bsky.social · 03/09/2026
Here’s a video showing how it looks for the user! With example survey questions at open and close
1144
Reposted by Jamie Cummins
Sabrina Norwood @sabrinanorwood.bsky.social · 03/09/2026
Early in this study I knew I’d want to run many more observational and intervention studies using this method. And I wanted to make it easier for myself and other researchers to do it So @ianhussey.mmmdata.io & I spent time developing an iOS app that can collect these data and deliver interventions
1223
Reposted by Jamie Cummins
Sabrina Norwood @sabrinanorwood.bsky.social · 02/09/2026
🚨 New preprint: In this paper we make the case for measuring social media use at the level of naturally occurring sessions (i.e., beginning when the app/site is opened and ending when it is closed). Across 7 days and 161 participants this gave us nearly 4,000 paired sessions of use 🧵
Depicts the data collection process. 1) participant opens TikTok, 2) open survey is prompted, 3) participant uses TikTok as normal, 4) participant closes TikTok, 5) close survey is prompted
25916
Reposted by Jamie Cummins
simine vazire @simine.com · 02/09/2026
Psych Science has retracted Ariely & Wertenbroch (2002). The second author requested the retraction. Both authors agreed to it. journals.sagepub.com/doi/full/10.... So grateful to @datacolada.bsky.social for these blog posts and the work behind them: datacolada.org/138 and datacolada.org/139 1/2
journals.sagepub.com
Sage Journals: Discover world-class research
Subscription and open access journals from Sage, the world's leading independent academic publisher.
426794
Reposted by Jamie Cummins
JsonGeller @jgeller1phd.bsky.social · 02/09/2026
Are preregs just a way for people to virtual signal now? Comparing pregregs to published papers is very sobering. Also shout out to @debruine.bsky.social and @lakens.bsky.social for metacheck and regcheck @ianhussey.mmmdata.io for making this easy!
5175
Jamie Cummins @jamiecummins.bsky.social · 02/09/2026
This distinction is critical for when we talk and think about “evaluating AI” for research tasks. When thinking about institutional use of AI in science, we should be thinking about designed (open-source) genAI tools, not retail genAI.
130
Reposted by Jamie Cummins
simine vazire @simine.com · 28/08/2026
Psych Science (the journal) is broader than you might think! Do you do descriptive research? Or research w/ underrepresented populations or cultural frameworks? Or research on AI/LLMs for psych? Or research testing lab findings in the real world? We have a special issue for that!
14931
Reposted by Jamie Cummins
Randy Ellis @randalljellis.bsky.social · 28/08/2026
I’m a big fan of RegCheck by @jamiecummins.bsky.social. I found some deviations in a paper the other day using it regcheck.app/result/8825b... I hope journals start integrating it. I also recall our thread about registered reports having better defense against this
regcheck.app
RegCheck
RegCheck is an AI tool to compare preregistrations with papers instantly.
1155
Jamie Cummins @jamiecummins.bsky.social · 27/08/2026
Four (4!) special issues at Psychological Science - including one I am co-editing on the use of LLMs in psychological science. Come submit your work!
1101
Reposted by Jamie Cummins
Tom Hardwicke @tomhardwicke.bsky.social · 27/08/2026
3. From Lab to Life: Psychological Science in the Real World www.psychologicalscience.org/publications... 4. Centering Underrepresented Populations and Cultural Frameworks in Psychological Science: African and Global Perspectives www.psychologicalscience.org/publications...
0129
Reposted by Jamie Cummins
Tom Hardwicke @tomhardwicke.bsky.social · 27/08/2026
Send your best work to Psychological Science! Four special issue calls: 1. Descriptive Research in Psychological Science www.psychologicalscience.org/publications... 2. Advancing Psychological Science Using Large Language Models www.psychologicalscience.org/publications...
12316
Reposted by Jamie Cummins
Julia M. Rohrer @dingdingpeng.the100.ci · 27/08/2026
Psychological Science special issue call 📣 Ever had your research denigrated bc it's "only descriptive", "what's the mechanism though?" and "but what are the theoretical implications?" Now has come your time to shine. Send your best descriptive work! www.psychologicalscience.org/publications...
Psychological Science invites submissions that advance psychological science through careful description of psychological phenomena. Psychological research often focuses on explaining and predicting phenomena before we have a sufficiently detailed understanding of them. It can be difficult to build strong theories on incomplete empirical foundations of the phenomena themselves. Careful descriptive research helps establish shared empirical reference points developed around well-characterized phenomena. As such, descriptive research contributes to psychological science both by improving our understanding of the phenomena themselves and by providing a stronger foundation for future theory-building, explanation, and prediction.

We are interested in manuscripts that:

Deepen our understanding of psychological phenomena, including what they look like, how they vary, and how they unfold;
Draw on a variety of sources of evidence and methods, including qualitative, quantitative, and mixed-methods approaches; research conducted in naturalistic settings; intensive, longitudinal, or small-sample designs; and open or large-scale datasets; 
Provide necessary context for interpreting what is being described, including who is represented in the data, the settings and circumstances in which phenomena occur, and the broader social contexts in which they are situated; and/or
Make a substantive contribution to psychological science through the quality, depth, or significance of the description provided.
The deadline for submitting regular manuscripts is April 15, 2027. Registered Reports will also be considered. The deadline for submitting Stage 1 Registered Reports is November 15, 2026. Authors should indicate in the “Comments to Editor” box during submission that the manuscript is intended for the “Descriptive Research in Psychological Science” special issue.
7244128
Reposted by Jamie Cummins
Beth Clarke @bethclarke.bsky.social · 21/08/2026
I’m hiring a PhD student to work with me on metascience in psychology at @zpid.bsky.social! The position is very open - could cover a wide range of topics/themes 🤸🏻‍♀️✨ If you have questions, please reach out! Applications are open until October 1st. leibniz-psychology.onlyfy.jobs/en/job/fyjwt...
leibniz-psychology.onlyfy.jobs
PhD in Psychological Meta-Science (m/f/d)
Become part of the open science movement The Leibniz Institute for Psychology Information (ZPID) in Trier (Germany) is an internationally recognized research-based psychology service center located in...
36157
Reposted by Jamie Cummins
Sung Kim @sungkim.bsky.social · 19/08/2026
Mathematics in the age of AI by Terence Tao An essay on how the mathematical community might respond to the arrival of AI tools that are capable of performing research-level mathematical tasks. arxiv.org/abs/2608.16753
arxiv.org
Mathematics in the age of AI
An essay, based on a public lecture delivered at the 2026 International Congress of Mathematicians, on how the mathematical community might respond to the arrival of artificial intelligence tools that...
16418
Reposted by Jamie Cummins
Association for Psychological Science @psychscience.bsky.social · 19/08/2026
Psychological Science invites submissions for a special issue: “Advancing Psychological Science Using Large Language Models” #LLMs Stage 1 Registered Reports due October 15. Regular articles due March 15. Questions? Ask the Editors: @jamiecummins.bsky.social and  @ruben.the100.ci
psychologicalscience.org
Special Issue Call for Submissions: “Advancing Psychological Science Using Large Language Models”
Editors: Jamie Cummins and Ruben C. ArslanPsychological Science invites submissions for a special issue on psychological science involving large language models. We welcome theoretically and empirical...
057
Reposted by Jamie Cummins
Isaiah Bishop @isaiahbishop.bsky.social · 19/08/2026
"if there was a cure to cancer they wouldnt sell it to you, to sell you chemo instead" (actual market): holy fuck these guys mightve cured melanoma BUY BUY BUY
241829362
Reposted by Jamie Cummins
Society for the Improvement of Psychological Science (SIPS) @improvingpsych.org · 17/08/2026
Congratulations to the project “RegCheck,” which received🏆2026 SIPS commendation! RegCheck is a tool that makes it easier and quicker for researchers to compare published papers with their associated study registrations. regcheck.app
regcheck.app
RegCheck
RegCheck is an AI tool to compare preregistrations with papers instantly.
13010
Reposted by Jamie Cummins
Saloni @scientificdiscovery.dev · 17/08/2026
It's finally here! I gave a TED talk in April in Vancouver, and it was released online today. You can watch it here: ted.com/talks/saloni... I've also adapted it into an article, which was published last week: worksinprogress.co/issue/future...
Me on the TED stage
612023
Reposted by Jamie Cummins
Nick DeVito @ndevito1.bsky.social · 17/08/2026
If you've got interesting research, experiences, or thoughts on open code in science (or elsewhere!) please consider putting in an abstract to this year's @bennettoxford.bsky.social Symposium! Info here: www.bennett.ox.ac.uk/blog/2026/08...
bennett.ox.ac.uk
Got something exciting to share? Submissions are open for the 2026 Bennett Oxford Symposium on Open Code in Science! | Bennett Institute for Applied Data Science
Call for submissions for the 2026 Bennett Oxford Symposium on Open Code in Science
001
Reposted by Jamie Cummins
Daniel Lakens @lakens.bsky.social · 14/08/2026
The Metacheck Shiny app to download all private OSF Repositories is done. The manual is updated, and everything is in the dev branch on GitHub. As a bonus, we explain how you can run repo_check, data_check, code_check, and codebook_check on your repository with 1 line of code. Launch next week.
14019
Jamie Cummins @jamiecummins.bsky.social · 13/08/2026
We're now accepting submissions for this year's @bennettoxford.bsky.social symposium - with the theme Open Code in Science! Submit now: www.bennett.ox.ac.uk/blog/2026/08...
bennett.ox.ac.uk
Got something exciting to share? Submissions are open for the 2026 Bennett Oxford Symposium on Open Code in Science! | Bennett Institute for Applied Data Science
Call for submissions for the 2026 Bennett Oxford Symposium on Open Code in Science
031
Reposted by Jamie Cummins
Daniel Lakens @lakens.bsky.social · 13/08/2026
My favorite Metascience conference will land in Copenhagen next year. The 9th (!) edition of the Perspectives on Science Error conference will be from January 27 to 29. Block your calendar, and see you there! errorsin.science
errorsin.science
Perspectives on Scientific Error 2027 · PSE9
Johanna Einsiedler, University of Copenhagen & University of Basel, johanna.einsiedler@sodas.ku.dk
12815
Reposted by Jamie Cummins
James Bartlett @bartlettje.bsky.social · 12/08/2026
We have a new pre-print, which I am very excited about! We report an exploratory mixed methods study to see how students would find wrangling realistic data as part of the analysis process. osf.io/preprints/ps...
osf.io
OSF
141
Reposted by Jamie Cummins
Society for the Improvement of Psychological Science (SIPS) @improvingpsych.org · 11/08/2026
The Center for Open Science recently announced its decision to phase out OSF Projects. We want to share how those changes will affect PsyArXiv, a psychology preprint server run and maintained by SIPS volunteers but hosted on the OSF platform. improvingpsych.org/2026/08/11/w...
improvingpsych.org
What Researchers Need to Know about PsyArXiv Given Upcoming Changes to OSF Projects
In just a few short days (August 15th, 2026), PsyArXiv – the community preprint server for psychology – turns 10! 🥳 Over the past decade, our community has published over 58,000 preprin…
02628
Reposted by Jamie Cummins
Saloni @scientificdiscovery.dev · 11/08/2026
BIG NEW piece by me. A few years ago, if you asked me to summarize my entire worldview in one article, I would’ve said it was too broad and too difficult. But this year, I decided to do it.
worksinprogress.co
Every disease is a policy failure
Few people know how much medicine has progressed in their lifetimes. Fewer know how much faster it could be progressing.
1327189