Lana Tikhomirov @lanatikhomirov.bsky.social · 06/08/2025Our main findings from the scoping review are: 1. Most silent trials do not report model metrics outside of AUC (rarely reporting bias testing, failure modes, and data drift) 2. The evaluator of the silent trial is often unnamed or underspecified- human factors and stakeholder engagement is rare 010
Lana Tikhomirov @lanatikhomirov.bsky.social · 06/08/2025Our preprint is up! Ever heard of the silent phase of AI evaluation for medical AI? Well, now we’ve summarised the current state of research! 031
Lana Tikhomirov @lanatikhomirov.bsky.social · 30/04/2025Interested in AI from the perspective of cognitive science? Listen to Nesh Nikolic from Strategic Psychology interview me on the Better Thinking Podcast as we discuss AI and Psychology! neshnikolic.com/podcast/lana... 041
Reposted by Lana TikhomirovParis Marx @parismarx.com · 09/02/2025Funny to see Western AI executives say the hype around DeepSeek is exaggerated when their entire industry is built on exaggeration of the capabilities and possible returns of Western generative AI.cnbc.comDeepseek’s AI model is ‘the best work’ out of China but the hype is 'exaggerated,' Google Deepmind CEO saysDeepseek's AI model "is probably the best work" out of China, Demis Hassabis said on Sunday, but added it was not a scientific advancement. 723538
Reposted by Lana TikhomirovDr Abeba Birhane @abeba.blacksky.app · 29/01/2025openAI’s data = data they illegally harvested from each and every one of us without consent, awareness or compensation 611328
Reposted by Lana TikhomirovJasmine 🌌🔭 @astrojaz.bsky.social · 28/01/2025i can’t fix anything that’s going on right now, but i can show you some beauty that’s out there… 🌌✨ 2139129910
Reposted by Lana TikhomirovMargaret Mitchell @mmitchell.bsky.social · 27/01/2025Holy moly. I'm trying to write an academic paper, and nearly every application I'm using is not only offering Generative AI as an option for writing, but *pushing it* -- pervading the design to the point where a simple misclick would make my content AI-generated. Here's why that's a problem. 🧵 31842312
Reposted by Lana TikhomirovHank Green @hankgreen.bsky.social · 28/01/2025The fact that Deepseek R1 was released three days /before/ Stargate means these guys stood in front of Trump and said they needed half a trillion dollars while they knew R1 was open source and trained for $5M. Beautiful. 396138071762
Reposted by Lana TikhomirovAmber Sparks @ambersparks.bsky.social · 07/01/2025Mark Zuckerberg: I built this platform to give the people a voice Me: okay but actually you built it to rate how hot the woman at your school were 864399584559
Reposted by Lana TikhomirovBen Harrap @bharrap.bsky.social · 07/01/2025Abstract submissions for the Global Indigenous Data Sovereignty (GIDSov) 2025 conference close on January 17th! See the conference website for themes and submission details gidsov.com.au/abstracts #IDSov #IDGov 051
Reposted by Lana TikhomirovMelissa McCradden @mdmccradden.bsky.social · 06/01/2025Huge thanks for the support from @aimlofficial.bsky.social and CIHR for their generous funding support! 041
Reposted by Lana TikhomirovMelissa McCradden @mdmccradden.bsky.social · 06/01/2025Anton van der Vegt Mjaye Mazwi @karinv.bsky.socialmedia.tenor.coma woman in a pink jacket is sitting at a table and laughing with the words whoop whoop written below her .ALT: a woman in a pink jacket is sitting at a table and laughing with the words whoop whoop written below her . 141
Reposted by Lana TikhomirovMelissa McCradden @mdmccradden.bsky.social · 06/01/2025Lesley-Anne Farmer Bobby Greer Anna Goldenberg Yvonne Ho @shalmalijoshi.bsky.social Jennie Louise Muhammad Mamdani Abdu Mohamud Lyle Palmer Antonios Peperidis Stephen Pfohl Mandy Rickard Carolyn Semmler @kdpsingh.bsky.social Devin Singh Seyi Soremekun @lanatikhomirov.bsky.social 121
Reposted by Lana TikhomirovMelissa McCradden @mdmccradden.bsky.social · 06/01/2025Super grateful to our amazing steering team, our patient/consumer partners, our partners at the Aboriginal Health Unit at Women’s and Children’s. 🙏🏻 @unityofvirtue.bsky.social Judy Gichoya Mark Sendak Lauren Erdman @istedman.bsky.social Lauren Oakden-Rayner Ismail Akrout James Anderson 121
Reposted by Lana TikhomirovMelissa McCradden @mdmccradden.bsky.social · 06/01/2025So reach out to me and @xiaoliu.bsky.social if you want to get involved with #CANAIRI to build the practices around translational trials for better AI translation! 131
Reposted by Lana TikhomirovMelissa McCradden @mdmccradden.bsky.social · 06/01/2025We propose that ethical governance for health institutions should be grounded in these local evaluations (not just AI vibes 😎) to ensure that when we say a tool ‘works,’ it means it works for US, for OUR patients, for OUR staff, and we have the evidence to say that.media.tenor.coma woman is standing on a balcony holding a piece of paper and saying i have the receipts .ALT: a woman is standing on a balcony holding a piece of paper and saying i have the receipts . 132
Reposted by Lana TikhomirovMelissa McCradden @mdmccradden.bsky.social · 06/01/2025What does this look like? It will be different for each tool, depending on many things including how much works has been done before on similar tools, how different the local context is, etc. How do we decide? That’s what we want to figure out, and we need your help! 121
Reposted by Lana TikhomirovMelissa McCradden @mdmccradden.bsky.social · 06/01/2025Fairness evaluations, human factors, cognitive science, patient engagement, environmental considerations (+++ to this one!), economics, and more ➡️ taking this global perspective from the get-go we think will reduce wasteful translation and optimize the benefit! 141
Reposted by Lana TikhomirovMelissa McCradden @mdmccradden.bsky.social · 06/01/2025Many know this already. But we can be doing better with how silent trials are practiced. It’s not just about the model - so we introduce ‘translational trial’ to signal the need for a more holistic evaluation during this silent stage, recognizing that AI is a sociotechnical toolmedia.tenor.coma man in a suit is standing in a doorway and saying `` what you 're thinking but way more ''ALT: a man in a suit is standing in a doorway and saying `` what you 're thinking but way more '' 141
Reposted by Lana TikhomirovMelissa McCradden @mdmccradden.bsky.social · 06/01/2025📣 CANAIRI: the Collaboration for Translational AI Trials! Co lead @xiaoliu.bsky.social @naturemedicine.bsky.social Perhaps most important to AI translation is the local silent trial. Ethically, and from an evidentiary perspective, this is essential! url.au.m.mimecastprotect.com/s/pQSsClx14m...url.au.m.mimecastprotect.com 1135
Reposted by Lana TikhomirovDr Abeba Birhane @abeba.blacksky.app · 23/12/2024"When trying to develop a measure of intelligence, it’s essential to avoid Goodhart’s law: “When a measure becomes a target, it ceases to be a good measure.” As a community of AI researchers, we really need to figure that one out." @melaniemitchell.bsky.social aiguide.substack.com/p/did-openai...aiguide.substack.comDid OpenAI Just Solve Abstract Reasoning?OpenAI’s o3 model aces the "Abstraction and Reasoning Corpus" — but what does it mean? 411234
Reposted by Lana TikhomirovOlivia Guest · Ολίβια Γκεστ @olivia.science · 13/12/2024yeah, exactly. anti-science correlation is cognition, busted logic doi.org/10.1007/s421... 0113
Reposted by Lana TikhomirovDeb Raji @rajiinio.bsky.social · 11/12/2024The really grim reality is that this is really not just about kids - there are countless adults, often adult *professionals*(doctors, lawyers, teachers(!)) that use ChatGPT this way. And I can understand why - there's no warning on the box! Few truly understand the tech's very real limitations. 915345
Reposted by Lana TikhomirovDr Abeba Birhane @abeba.blacksky.app · 10/12/2024I'd like people to stop using the term alignment (as in aligning AI with human values). it's just vacuous. human value is infinite & culture, history, ideology & context dependent & not something you can entirely quantify & automate either 814842
Reposted by Lana TikhomirovIris van Rooij 💭 @irisvanrooij.bsky.social · 06/12/2024“Few people realize that cognitive science is crucial for evaluating claims about AI capabilities. We often overestimate what computers are capable of, while vastly underestimating what human cognition is capable of.” www.ru.nl/en/research/...ru.nlDon’t believe the hype: AGI is far from inevitable | Radboud UniversityWill AI soon surpass the human brain? If you ask employees at OpenAI, Google DeepMind and other large tech companies, it is inevitable. However, researchers at Radboud University and other institutes ... 9427151
Lana Tikhomirov @lanatikhomirov.bsky.social · 03/12/2024Blog post: 'The human-AI relationship in medicine- why should we care?' medium.com/@aiml_58187/...medium.comThe human-AI relationship in medicine — why should we care?By Lana Tikhomirov 072
Lana Tikhomirov @lanatikhomirov.bsky.social · 03/12/2024Some examples of my work include this conceptual piece in the Lancet DH: www.thelancet.com/journals/lan...thelancet.comMedical artificial intelligence for clinicians: the lost cognitive perspectiveThe development and commercialisation of medical decision systems based on artificial intelligence (AI) far outpaces our understanding of their value for clinicians. Although applicable across many fo... 161
Lana Tikhomirov @lanatikhomirov.bsky.social · 03/12/2024Shoutout to my fantastic supervisors @carosemm.bsky.social, @mdmccradden.bsky.social and Lauren Oakden-Rayner 000
Lana Tikhomirov @lanatikhomirov.bsky.social · 03/12/2024Hey everyone! Thought I'd introduce myself to any new followers here- I'm a PhD student at the University of Adelaide with a background in cognitive psychology, specialising in high-risk decision-making with AI algorithms. I also use perspectives from bioethics and algorithmic safety in my work. 0150
Reposted by Lana TikhomirovPessoaBrain @pessoabrain.bsky.social · 29/11/2024𝐂𝐥𝐚𝐫𝐢𝐭𝐲 𝗮𝗿𝗼𝘂𝗻𝗱 𝗰𝗮𝘂𝘀𝗮𝗹𝗶𝘁𝘆 𝗶𝗻 𝗻𝗲𝘂𝗿𝗼𝘀𝗰𝗶𝗲𝗻𝗰𝗲 Causation is central to thinking in #neuroscience but is a complex, nuanced issue. We wrote this piece in 2022 having in mind expansions to full papers but not there yet. But definitely an important question for conceptual development doi.org/10.1016/j.ti... 611338
Reposted by Lana TikhomirovJessica Hullman @jessicahullman.bsky.social · 27/11/2024Wait, are the AnthropicAI people seriously claiming to “unlock a rich theoretical landscape” for AI evaluation by proposing the use of…. error bars? And this secret trove of deep statistical insight starts with “use the Central Limit Theorem”? Befuddling 119715