Sign in

Ingo Rohlfing

@ingorohlfing.bsky.social
4.5K followers 1.4K following 1.5K posts

I am here for all interesting and funny posts on the social sciences, broadly understood and including open science and meta science, academia, teaching and research. linktr.ee/ingorohlfing

PostsRepliesMedia
Reposted by Ingo Rohlfing
Vincent Arel-Bundock @vincentab.bsky.social · 14/10/2026
Hi all! I'm looking for people to try out my new collaborative writing platform. Think: Overleaf or GDocs for Mkdown LaTeX HTML Typst Quarto Free, open source, and full-featured. Many editors, chat, comments, track changes, history, AI, mobile, templates, etc Would ❤️ any feedback! librepaper.org
3025591
Reposted by Ingo Rohlfing
Danbischof @danbischof.bsky.social · 02/10/2026
Postdoc position within my group 3 years (100% TVl-13): stellen.uni-muenster.de/jobposting/2... ⏱️ 06.11.2026 If you have questions: get in touch!
stellen.uni-muenster.de
Postdoctoral Research Associate (Wissenschaftliche*r Mitarbeiter*in, salary level E 13 TV-L)
15851
Ingo Rohlfing @ingorohlfing.bsky.social · 14/10/2026
So, you fell off the causal ladder? I guess I understand it, but also do not fully follow the arguments, in particular on rung 2 and 3.
130
Ingo Rohlfing @ingorohlfing.bsky.social · 13/10/2026
pedagogically helpful. One can do this by drawing random samples from population data. This comes with the caveat that one does not have population data in practice and cannot resample multiple times, but there is no easy way to teach frequentism, is there? 3/
000
Ingo Rohlfing @ingorohlfing.bsky.social · 13/10/2026
A second reason was a concern that it would be too demanding in an intro course. Frequentism and repeated sampling from a population is hard enough, resampling from a sample would take it too far in an introductory class. This means I do not share the article's view that rebranding the sample as 2/
110
Ingo Rohlfing @ingorohlfing.bsky.social · 13/10/2026
When the Sample “Becomes” the Population: Shifting Worlds With the Bootstrap onlinelibrary.wiley.com/doi/full/10.... Does anyone teach bootstrapping in (applied) intro to statistics? I once considered it because the idea is intriguing, but there was not enough space in the curriculum. 1/
3112
Ingo Rohlfing @ingorohlfing.bsky.social · 13/10/2026
Just saw that a Wiley journal posted the URL for the reproduction material of an article using a linkshortener. This seems unneccessary and may not age well.
Screenshot from webpage stating: "The source R code for the output presented in this paper can be found here: https://bit.ly/bootstrap20260601."
060
Reposted by Ingo Rohlfing
Julia M. Rohrer @dingdingpeng.the100.ci · 13/10/2026
”Using vibe-coded, unvalidated forensic meta-science tools to evaluate the trustworthiness of published research risks holding authors vs. tools to unequal standards. The solution is less vibe-coding and more validation.”
23411
Reposted by Ingo Rohlfing
Michael P. Grosz @mp-grosz.bsky.social · 12/10/2026
Our new paper has just been accepted at AMPPS 😊 In psychology, we often bundle different things into composites (extraversion, depression, …). If the parts have different effects, the composite's effect is an ambiguous blend. Above all, this is Lukas Junker's achievement. osf.io/preprints/ps...
osf.io
Consistency: A Critical but Often Overlooked Requirement for Causal Inference in Psychological Research
Psychologists are often interested in specific causal quantities (“What is the effect of reducing social-media scrolling by 20 minutes on well-being?”) but tend to focus on vague questions (“Does ‘tim...
23013
Ingo Rohlfing @ingorohlfing.bsky.social · 10/10/2026
Not quite what is asked for, but this maybe going in the right direction. I haven't checked whether this was published in a journal in the meantime. Protzko, John. 2018. “Null-Hacking, a Lurking Problem.” doi:10.31234/osf.io/9y3mp.
000
Reposted by Ingo Rohlfing
Dan Quintana @dsquintana.bsky.social · 09/10/2026
Publication bias is the idea that non-significant results are less likely to be published. But what about the opposite case where a *significant* result is less likely to be published? Anyone have any papers or tools for detecting and/or correcting for “reverse” publication bias?
983
Reposted by Ingo Rohlfing
Alex Holcombe @alexh.bsky.social · 09/10/2026
With more and more websites going down, whether overwhelmed by bot traffic or starved of ad revenue, it is time for scholarly referencing guidelines for citing a website to include an archived version, just as Wikipedia has long recommended en.wikipedia.org/wiki/Wikiped... #metascience
en.wikipedia.org
Wikipedia:Archiving a source - Wikipedia
2239
Reposted by Ingo Rohlfing
Michael Salmon @michaelsalmon0.bsky.social · 09/10/2026
Important explainer-slash-mission statement from the metascience unit on the use of AI in research evaluation. Read all about it on Research Agenda: www.researchagenda.news/articles/the...
researchagenda.news
The metascience unit sets out its thinking on AI-powered research evaluation
A basket of indicators will support a “new vocabulary” of research quality
002
Ingo Rohlfing @ingorohlfing.bsky.social · 08/10/2026
The resulting interval for power is also called a confidence interval. I do not see it as a confidence interval because it is defined as the range of plausible population values (not discussing "plausible" here). Power of a test is not a population value, so how can it have a conf int attached? 2/
000
Ingo Rohlfing @ingorohlfing.bsky.social · 08/10/2026
A Post-Hoc Power Primer - Replicability-Index replicationindex.com/2025/03/22/a... Leaving aside that post-hoc power analysis is contested: I see the rationale for estimating power for the lower and upper bound of the point estimate's confidence interval rather than only using the point est. 1/
101
Reposted by Ingo Rohlfing
Alondra Nelson @alondra.bsky.social · 10/09/2026
Researchers across the country are turning to side hustles to fund their work. Behind the cuts sits a bigger shift: whether American science can still sustain durable research capacity. @lindseydcameron.bsky.social and I examine the stakes in a new piece @science.org: www.science.org/doi/10.1126/...
science.org
The wrong platform for science
Daniel Blumstein recently sat in a Colorado meadow waiting for yellow-bellied marmots to appear, as the ecologist and evolutionary biologist had done many times before, the New York Times reported. Bu...
7529283
Reposted by Ingo Rohlfing
Eiko Fried @eikofried.bsky.social · 11/09/2026
Ever since this happened, I have never given journals sole ownerhip of my supplemental materials .. eiko-fried.com/blog/supplem...
eiko-fried.com
Supplementary materials fail | Eiko Fried
A few months ago, we published a new paper Fried, E. I., van Borkulo, C. D., Cramer, A. O. J., Boschloo, L., Schoevers, R. A., & Borsboom, D. (2017). Mental disorders as networks...
96618
Reposted by Ingo Rohlfing
European Consortium for Political Research @ecpr.bsky.social · 07/10/2026
▶️ Start your next project with the right #ResearchMethods 👀 Get a first look at our Methods School winter programme and sign up for ✨️ early access ✨️ to book your preferred course 48 hours before general sale buff.ly/SrBFl7q 📆 15–26 Feb 2027, Online #ecprms #methodsky #polisky
133
Reposted by Ingo Rohlfing
Alexander Wuttke @kunkakom.bsky.social · 06/10/2026
GESIS Spring Seminar 2027 on Metascience: Methods and Tools for Robust Inference t1p.de/Spring2027
042
Ingo Rohlfing @ingorohlfing.bsky.social · 06/10/2026
Chinese-owned science journal created to rival the best www.nature.com/articles/d41... An incentive for non-China based researchers is that they are not subject to APCs. If interest is to define what counts as high-quality research, then this smells like interference in the editorial process.
000
Reposted by Ingo Rohlfing
A Jay Holmgren @ajh.bsky.social · 05/10/2026
Discousre on this paper www.nature.com/articles/s41... on the site has convinced me that we desperately need a metascience movement for the metascientists.
nature.com
Investigating the replicability of the social and behavioural sciences - Nature
A large-scale study on the replicability of claims from social and behavioural science journals reports that about half of the results replicate in the same patterns as the original study.
1308
Reposted by Ingo Rohlfing
Berna Devezer @devezer.bsky.social · 05/10/2026
I don't understand this. 52 years later, why try and replicate the original? It's obviously not going to be the same study. The world has changed; cars, traffic, and people have changed. If one is interested in false memory, why not design the best study you can? What can you conclude meaningfully?
79113
Ingo Rohlfing @ingorohlfing.bsky.social · 03/10/2026
journal to publish in may explain why the effect sizes of being a focal OA author rather than a non-focal author are small (I would say they are). 3/
000
Ingo Rohlfing @ingorohlfing.bsky.social · 03/10/2026
Focal authors are defined as having published at least three papers with a positive stance toward open-access publishing. If focal authors do not publish on their own, we don't know what considerations went into the authors' decision for a journal. Decision-making among the authors about the 2/
100
Ingo Rohlfing @ingorohlfing.bsky.social · 03/10/2026
Do #OpenScience advocates walk the talk? Evidence from positive position papers royalsocietypublishing.org/rsos/article... The evidence is mixed, illustrating the role of funder mandates and other factors. I wouldn't read too much into the comparison of focal authors and matched authors. 1/
Proportion of OA types among papers focused on open science by attitudinal stance - positive, neutral, negative (2014–2023).
110
Ingo Rohlfing @ingorohlfing.bsky.social · 02/10/2026
AI bots are flooding researchers with requests for money and time www.nature.com/articles/d41... "time" meaning agents are offering to collaborate with researchers. This seems peculiar because it is not clear who is behind an agent. Also, why agree to an invite when one can set up one's own agent?
163
Reposted by Ingo Rohlfing
moin syed @syeducation.bsky.social · 01/10/2026
Gave a talk in my dept. on the OSF transition. I wrote up a rough script of the session, which could be of interest to others. I synthesize existing resources and provide my own perspectives. Comments and corrections welcome! zenodo.org/records/2306...
the header for a paper describing "Life after OSF" #metascience
215872
Reposted by Ingo Rohlfing
Paquito Bernard @paquitobernard.bsky.social · 02/10/2026
Slow #epidemiology 'This means clearer questions, more rigorous designs, more transparent reporting, and more thorough sensitivity analyses. In other words, embracing the cutting edge of practice, not regurgitating the past.' #research #AI #health zenodo.org/records/2308...
zenodo.org
Why epidemiology should not be automated: the case for Slow Epidemiology
EXTRACT:Bann et al. pose the question, ‘why can’t epidemiology be automated (yet)?’, as though the automation of epidemiology is both inevitable and desirable [1]. They describe a future of ‘1-click r...
0114
Ingo Rohlfing @ingorohlfing.bsky.social · 29/09/2026
Excel now supports multiple values in a single cell | Microsoft Community Hub techcommunity.microsoft.com/blog/excelbl... Who wouldn't love an additional challenge in processing spreadsheets?
083
Ingo Rohlfing @ingorohlfing.bsky.social · 29/09/2026
NIH Launches $174M for Replicability and Reproducibility Initiatives www.insidehighered.com/news/governm... #MetaScience Includes funding lines that follow the many-labs idea (many being a handful here), install "rigor ambassadors" and an U.S.-based reproducibility network
051
Reposted by Ingo Rohlfing
J Dierkes @dierkes.bsky.social · 29/09/2026
We're hiring at School of Social Sciences at Univ of Mannheim Full Professorship (W-3) in Social Data Science and Social Science Methodology Apply by Oct 22 2026 www.sowi.uni-mannheim.de/en/about/job... @methodsnet.bsky.social
sowi.uni-mannheim.de
Job Vacancies
01831
Ingo Rohlfing @ingorohlfing.bsky.social · 28/09/2026
What would have been the alternative to publishing with Routledge according to the person who asked? Self-publishing for free, for example?
100
Ingo Rohlfing @ingorohlfing.bsky.social · 28/09/2026
I see the point, but not addressing the issue and rerunning the analysis could easily invite public charges that researcher xy is not interested in getting the record straight, is immune to criticism and so on.
200
Ingo Rohlfing @ingorohlfing.bsky.social · 28/09/2026
I guess these actors could have done this easily at any time, given their resources. But I see that LLMs and agents make this easier and that this can be weaponized by any organization or individual now who wants to drown researchers in messages.
100
Ingo Rohlfing @ingorohlfing.bsky.social · 28/09/2026
Apologies, it was @ruben.the100.ci bsky.app/profile/rube...
000
Ingo Rohlfing @ingorohlfing.bsky.social · 27/09/2026
Good question. I guess we will see this much more often now as it becomes easier to automate the LLM-based screening of repos at scale. I think one has to follow up on such an email. The sender does not show it matters, but neither can you show it does not matter unless you addressed it.
120
Ingo Rohlfing @ingorohlfing.bsky.social · 27/09/2026
I think @markrubin.bsky.social reported something similar on a larger scale earlier this month.
310
Ingo Rohlfing @ingorohlfing.bsky.social · 26/09/2026
Evidently, this is a bad workflow and one should just not let LLMs near citations and reference management. However, I think this is how some researchers work. 4/
110
Ingo Rohlfing @ingorohlfing.bsky.social · 26/09/2026
However, I can imagine that this can also happen without malicious intent. Like someone writes in a manuscript Pepinsky (2012) and then tells a badly managed LLM to add a reference for this citation, knowing that Pepinsky (2012) exists, yet the LLM invents some other paper. 3/
110
Ingo Rohlfing @ingorohlfing.bsky.social · 26/09/2026
this is relatively easy to detect. Now more than ever because more and more publishers are installing reference checks. Nonetheless, I believe there are researchers who just infuse some citations into their papers with an LLM because of the assumption that noone will care or check them. 2/
110
Ingo Rohlfing @ingorohlfing.bsky.social · 26/09/2026
There is no excuse for a fake citation. Authors are responsible for their papers, including citations and references, and fake citations should be sanctioned by an immediate desk rejection. Beyond that, I guess I am a bit lost why fake citations happen. It seems foolish to fake citations because 1/
100
Ingo Rohlfing @ingorohlfing.bsky.social · 25/09/2026
I think it is different in German law where "Betrug" requires intent. In any case, if it is legally fraud in the US, then it is fraud, but I still uphold that one cannot assume intent without evidence for it. What would you then suggest as a sanction if non-existent references are part of a paper?
100
Ingo Rohlfing @ingorohlfing.bsky.social · 25/09/2026
Again, it would be grossly negligent because LLM-based code should be validated and the paper should be corrected or retracted. Beyond that, I don't see a basis for inferring that it was fraud. 4/
100
Ingo Rohlfing @ingorohlfing.bsky.social · 25/09/2026
For me, it would be the same with a coding error. If there is a coding error, maybe LLM-coded, that produces an intended result that disappears w/o the error, was the coding error not an error but intentional to achieve this result knowingly? This is impossible to tell from the outside. 3/
100
Ingo Rohlfing @ingorohlfing.bsky.social · 25/09/2026
If an LLM is used for whatever reason and one does not validate the references, this is at least grossly negligent because everyone should know that inexistent references maybe added. The burden of proof that it was done intentionally rests with the editor and I don't see how one could do this. 2/
100
Ingo Rohlfing @ingorohlfing.bsky.social · 25/09/2026
I am, again, with Sebastian here. bsky.app/profile/adam... Maybe we understand "intent" differently. The overall question maybe is: Why would one use an LLM for citations and reference management in the first place? I can't think of a good usecase. 1/
220
Reposted by Ingo Rohlfing
John Stott @jpsastro.bsky.social · 24/09/2026
Had a UKRI funded grant proposal rejected at the "AI triage stage". "This AI-based triage was used only to identify approximately 50% of the strongest proposals to take forward to full human review." 🧪 #academicsky
I am writing to let you know that, unfortunately, your application was not successful.

Review process

As part of the initial triage of the 179 proposals submitted to this call, each was assessed using an AI-based review. Every proposal was read independently by several different AI models against the same seven criteria used to shape this call (including cyber security relevance, research quality and novelty, importance of the problem addressed, feasibility of the project plan, likely outputs and impact, value for money, and responsible research practice), with each model asked to give a score and a written justification with supporting quotations from the text. To guard against any one model's idiosyncrasies, we also ran a second, independent process in which different AI models debated the merits and weaknesses of each proposal before reaching a judgement, again against the same criteria. Scores from both processes were combined to produce an overall ranking. 

This AI-based triage was used only to identify approximately 50% of the strongest proposals to take forward to full human review.  Projects selected for funding were drawn from those that progressed to the full human review stage. We recognise that AI-assisted assessment is a new and evolving part of the review process, and we are continuing to evaluate how well it aligns with the judgements of human reviewers on this call. We will soon produce a document detailing our method, so that others can build upon it and improve it.

Feedback on your application

Your application was not selected to progress beyond the AI triage stage.  To provide transparency on the outcome of this assessment, we are sharing below the scores (out of 5) from each of the two AI review processes described above, together with the mean score across the two processes.
40435241
Ingo Rohlfing @ingorohlfing.bsky.social · 25/09/2026
This may still not be fraud because of lack of intent but I think it is more than error or an honest mistake. Everyone knows that the use of LLMs comes with the disclaimer that stuff may be made up. 2/
100
Ingo Rohlfing @ingorohlfing.bsky.social · 25/09/2026
I see it like others that the "knowingly" part would be almost impossible to hold up if charges of fraud would be raised. However, one may soften it by saying that by using LLMs, one should have known that they may make up references and that particular care is in order before submitting. 1/
110
Ingo Rohlfing @ingorohlfing.bsky.social · 25/09/2026
If the desk-rejection is only motivated by the false refs, I think it is better to set an incentive for revising them, hence the invite for a resubmission. 2/
000