iiinova — where your digital self keeps evolving

Reposted by Karin Verspoor

John Skiles Skinner @skiles.blue · 21/03/2025

I helped build a government AI system. DOGE fired me, rolled the AI out to the whole agency, and implied the AI can do my job and the jobs of the others they've fired. It can't. But, what DOGE accidentally revealed about themselves in the process is fascinating. 🧵

191103874334

Karin Verspoor @karinv.bsky.social · 22/03/2025

What are we looking at on the screen?

220

Karin Verspoor @karinv.bsky.social · 14/03/2025

Will share when it’s up. In the meantime here is something relevant to the use of LLMs for quality assessment of medical research: “Zero- and few-shot prompting of generative large language models provides weak assessment of risk of bias in clinical trials” doi.org/10.1002/jrsm...

doi.org

Zero‐ and few‐shot prompting of generative large language models provides weak assessment of risk of bias in clinical trials

Existing systems for automating the assessment of risk-of-bias (RoB) in medical studies are supervised approaches that require substantial training data to work well. However, recent revisions to RoB...

030

Karin Verspoor @karinv.bsky.social · 13/03/2025

Given the challenges that LLMs have in processing numbers correctly, and the prevalence and importance of numbers in scientific papers, I suspect this will not work well. We have a paper upcoming at NAACL’25 in May showing the limitations of LLMs for numerical reasoning.

100

Karin Verspoor @karinv.bsky.social · 09/03/2025

I made the shift from long commute on train+tram to an 8-minute walk. Highly recommend! I can even pop home for lunch!

050

Reposted by Karin Verspoor

Anne Charmantier @annecharmantier.bsky.social · 08/03/2025

Today is International Women's Day 👭🧪🔭. With this video we hope to motivate girls to engage in scientific careers, please circulate/ Cette vidéo en français/anglais a vocation à être diffusée dans les classes 👩‍👩‍👧‍👦. www.youtube.com/watch?v=uKlT... #InternationalWomensDay2025 #JourneeDesDroitsDesFemmes

youtube.com

Portraits de femmes scientifiques

YouTube video by FemmesetSciences

02822

Karin Verspoor @karinv.bsky.social · 08/03/2025

All 3 papers were sent to me by (different) @springernature.com "Discover" journals. Obviously they are being targeted by desperate authors. Stronger editorial pre-screening is needed.

010

Karin Verspoor @karinv.bsky.social · 08/03/2025

Another day, another completely incomprehensible paper to review, with no meaningful research question or results. More inappropriate references. The lit review was almost funny (if it weren't so bad) -- presented in alphabetical order by author's last name! When would that ever be a good idea?!

110

Karin Verspoor @karinv.bsky.social · 07/03/2025

I’m sure it is happening all the time, sadly. My concern is how often such papers are getting through peer review and getting published.

000

Karin Verspoor @karinv.bsky.social · 06/03/2025

@retractionwatch.com How do we avoid getting these papers published in the first place?!

020

Karin Verspoor @karinv.bsky.social · 06/03/2025

Update: The second paper received a “revise” decision, despite clear violations of editorial policies I listed in my review. I have contacted the EIC to request a review of that decision. NB none of the other reviewers commented on the fabricated and inappropriate references. The system is broken.

370

Karin Verspoor @karinv.bsky.social · 02/03/2025

For those of us who don’t exclusively publish in or need to read biomedical journals, Google Scholar provides an important (necessary) resource. I try to teach my students to scrutinise both the source and the quality of the work.

100

Karin Verspoor @karinv.bsky.social · 02/03/2025

So it is a matter of time investment, not innovation? Interesting.

010

Karin Verspoor @karinv.bsky.social · 02/03/2025

Thanks; Google Scholar is also up so I can find most things. I guess my concern is what's going on with PubMed!

320

Karin Verspoor @karinv.bsky.social · 02/03/2025

I just reviewed two papers. One was almost certainly written with heavy assistance from genAI. The other was full of missing, misleading and incorrect references, many apparently planted. I suspected as much from the abstracts. I hate that I have to spend time policing science. #researchintegrity

1204

Karin Verspoor @karinv.bsky.social · 02/03/2025

Super excited to be a part of this collaboration!

110

Karin Verspoor @karinv.bsky.social · 16/12/2024

This #AI image from an unknown source at an unknown time has been making the rounds and finally got to me. It is amusing because of how many ways it is wrong ... For me, it reminded me of challenges we faced working on references in recipes. Recipes are hard! aclanthology.org/2022.finding...

1112

Karin Verspoor @karinv.bsky.social · 15/03/2024

No, they just want people to cite it. That's different.

020

Karin Verspoor @karinv.bsky.social · 15/03/2024

@gcabanac.cpesr.fr in case you have not yet seen this, it is up your alley.

120

Karin Verspoor @karinv.bsky.social · 13/09/2023

I always use train-DEVELOPMENT-test I suppose because of the same discomfort. Validate is arguably more common the dev step. I suppose because we use it to validate choices during training? My guess is it is in a book/influential tutorial somewhere that way and that drove everything…

010