Sign in

Berk Ustun

@berkustun.bsky.social
2.7K followers 455 following 43 posts

Assistant Prof at UCSD. I work on safety, interpretability, and fairness in machine learning. www.berkustun.com

PostsRepliesMedia
Reposted by Berk Ustun
Lauren Dobson-Hughes @ldobsonhughes.bsky.social · 09/11/2025
UK government project using AI to find benefit fraud resulted in: - A 46% false fraud rate - Anguish for families who were wrongly accused of fraud and had benefits stopped - Months of additional work for government, setting up a hotline, correcting false fraud www.theguardian.com/society/2025...
theguardian.com
HMRC trial of child benefit crackdown wrongly suspected fraud in 46% of cases
Exclusive: Almost half of families flagged as emigrants based on Home Office travel data were still living in UK
1213691
Reposted by Berk Ustun
Jessica Hullman @jessicahullman.bsky.social · 22/10/2025
I’m giving an IDE seminar at @mitsloan.bsky.social tomorrow at 11am, on optimizing AI as decision support. Joint work w/ @ziyang.bsky.social @yifanwu.bsky.social @jasonhartline.bsky.social @berkustun.bsky.social Come by if you’re around! www.eventbrite.com/e/fall-2025-...
0114
Reposted by Berk Ustun
Loris D'Antoni @lorisdanto.bsky.social · 23/09/2025
Who teaches an undergraduate principles of programming languages class? Looking for some inspiration to teach one at UCSD
261
Reposted by Berk Ustun
Ben Recht @beenwrekt.bsky.social · 08/09/2025
In a new paper, I try to resolve the counterintuitive evidence of Meehl’s “clinical vs statistical prediction” problems: Statistics only wins because the game is rigged.
arxiv.org
The Actuary's Final Word on Algorithmic Decision Making
Paul Meehl's foundational work "Clinical versus Statistical Prediction," provided early theoretical justification and empirical evidence of the superiority of statistical methods over clinical judgmen...
3299
Reposted by Berk Ustun
Connor Lawless @lawlessopt.bsky.social · 14/07/2025
Machine learning models can assign fixed predictions that preclude individuals from changing their outcome. Think credit applicants that can never get a loan approved, or young patients that can never get an organ transplant - no matter how sick they are!
111
Reposted by Berk Ustun
Connor Lawless @lawlessopt.bsky.social · 14/07/2025
Excited to be chatting about our new paper "Understanding Fixed Predictions via Confined Regions" (joint work with @berkustun.bsky.social, Lily Weng, and Madeleine Udell) at #ICML2025! 🕐 Wed 16 Jul 4:30 p.m. PDT — 7 p.m. PDT 📍East Exhibition Hall A-B #E-1104 🔗 arxiv.org/abs/2502.16380
arxiv.org
Understanding Fixed Predictions via Confined Regions
Machine learning models can assign fixed predictions that preclude individuals from changing their outcome. Existing approaches to audit fixed predictions do so on a pointwise basis, which requires ac...
153
Reposted by Berk Ustun
Jessica Hullman @jessicahullman.bsky.social · 02/07/2025
Paper: www.arxiv.org/abs/2506.22740 Blog post: statmodeling.stat.columbia.edu/2025/07/02/w...
arxiv.org
Explanations are a means to an end
Modern methods for explainable machine learning are designed to describe how models map inputs to outputs--without deep consideration of how these explanations will be used in practice. This paper arg...
2233
Reposted by Berk Ustun
Jessica Hullman @jessicahullman.bsky.social · 02/07/2025
ExplainableAI has long frustrated me by lacking a clear theory of what an explanation should do. Improve use of a model for what? How? Given a task what's max effect explanation could have? It's complicated bc most methods are functions of features & prediction but not true state being predicted 1/
2468
Reposted by Berk Ustun
Maria De-Arteaga @mariadearteaga.bsky.social · 25/06/2025
Having a lot of FOMO not being able to be in person at #FAccT2025 but enjoying the virtual transmission 💻. Tomorrow Jakob will be presenting our paper "Perils of Label Indeterminacy: A Case Study on Prediction of Neurological Recovery After Cardiac Arrest".
screenshot of title and authors (Jakob Schoeffer, Maria De-Arteaga, Jonathan Elmer)
1131
Berk Ustun @berkustun.bsky.social · 24/06/2025
Explanations don't help us detect algorithmic discrimination. Even when users are trained. Even when we control their beliefs. Even under ideal conditions... 👇
0110
Reposted by Berk Ustun
Peter Sokol-Hessner @p1sh.bsky.social · 26/04/2025
“Science is a smart, low cost investment. The costs of not investing in it are higher than the risk of doing so… talk to people about science.” - @kevinochsner.bsky.social makes his case to the field #sans2025
616955
Reposted by Berk Ustun
sorelle @friedler.net · 24/04/2025
Hey AI folks - stop using SHAP! It won't help you debug [1], won't catch discrimination [2], and makes no sense for feature importance [3]. Plus - as we show - it also won't give recourse. In a paper at #ICLR we introduce feature responsiveness scores... 1/ arxiv.org/pdf/2410.22598
Left: a feature-highlighting explanation generated by SHAP that shows multiple important features, however these include features that can not be changed (e.g., age, number of dependents) and features that even if they were changed would not result in a different outcome (e.g., credit utilization).

Right: a feature-highlighting explanation generated by our responsiveness scores showing only features that can be changed and which have the potential to result in a better outcome for the individual (multiple credit lines and monthly income).
3288
Reposted by Berk Ustun
Hailey Joren @haileyjoren.bsky.social · 24/04/2025
When RAG systems hallucinate, is the LLM misusing available information or is the retrieved context insufficient? In our #ICLR2025 paper, we introduce "sufficient context" to disentangle these failure modes. Work w Jianyi Zhang, Chun-Sung Ferng, Da-Cheng Juan, Ankur Taly, @cyroid.bsky.social
1115
Reposted by Berk Ustun
Harry Cheon @scheon.com · 24/04/2025
Denied a loan, an interview, or an insurance claim by machine learning models? You may be entitled to a list of reasons. In our latest w @anniewernerfelt.bsky.social @berkustun.bsky.social @friedler.net, we show how existing explanation frameworks fail and present an alternative for recourse
1157
Reposted by Berk Ustun
Karl Rohe @karlrohe.bsky.social · 19/04/2025
Absolute banger.
4304
Reposted by Berk Ustun
Sujay Nagaraj @snagaraj.bsky.social · 20/04/2025
Many ML models predict labels that don’t reflect what we care about, e.g.: – Diagnoses from unreliable tests – Outcomes from noisy electronic health records In a new paper w/@berkustun, we study how this subjects individuals to a lottery of mistakes. Paper: bit.ly/3Y673uZ 🧵👇
1122
Reposted by Berk Ustun
Sujay Nagaraj @snagaraj.bsky.social · 13/04/2025
🚨 Excited to announce a new paper accepted at #ICLR2025 in Singapore! “Learning Under Temporal Label Noise” We tackle a new challenge in time series ML: label noise that changes over time 🧵👇 arxiv.org/abs/2402.04398
arxiv.org
Learning under Temporal Label Noise
Many time series classification tasks, where labels vary over time, are affected by label noise that also varies over time. Such noise can cause label quality to improve, worsen, or periodically chang...
151
Reposted by Berk Ustun
Mark Riedl @markriedl.bsky.social · 05/02/2025
The CHI Human-Centered Explainable AI Workshop is back! Paper submissions: Feb 20 hcxai.jimdosite.com
hcxai.jimdosite.com
Home | HCXAI
ACM CHI 2025 Workshop on Human-Centered Explainable AI (HCXAI). May 2025 (Yokohama, Japan & hybrid). Submit your Paper (EasyChair)
02612
Reposted by Berk Ustun
logan koepke @jlkoepke.bsky.social · 21/01/2025
🧵on the CFPB and less discriminatory algorithms. last week, in its supervisory highlights, the Bureau offered a range of impressive new details on how financial institutions should be searching for less discriminatory algorithms.
1165
Reposted by Berk Ustun
Jacobs Center for Health Innovation @UC San Diego Health @jchi-ucsd.bsky.social · 08/01/2025
Engaging discussions on the future of #AI in #healthcare at this week's ICHPS, hosted by @amstatnews.bsky.social. JCHI's @kdpsingh.bsky.social shared insights on the safety & equity of #MachineLearning algorithms and examined bias in large language models.
194
Berk Ustun @berkustun.bsky.social · 07/01/2025
Safety that matters www.ftc.gov/policy/advoc...
ftc.gov
AI and the Risk of Consumer Harm
People often talk about “safety” when discussing the risks of AI causing harm. AI safety means different things to different people, and those looking for a definition here will be disappointed.
070
Reposted by Berk Ustun
Melissa McCradden @mdmccradden.bsky.social · 06/01/2025
📣 CANAIRI: the Collaboration for Translational AI Trials! Co lead @xiaoliu.bsky.social @naturemedicine.bsky.social Perhaps most important to AI translation is the local silent trial. Ethically, and from an evidentiary perspective, this is essential! url.au.m.mimecastprotect.com/s/pQSsClx14m...
url.au.m.mimecastprotect.com
1135
Reposted by Berk Ustun
Benjamin Laufer @laufer.bsky.social · 13/12/2024
🪩New paper🪩 (WIP) appearing at @neuripsconf.bsky.social Regulatable ML and Algorithmic Fairness AFME workshop (oral spotlight). In collaboration with @s010n.bsky.social and Manish Raghavan, we explore strategies and fundamental limits in searching for less discriminatory algorithms.
282
Reposted by Berk Ustun
Jessica Hullman @jessicahullman.bsky.social · 10/12/2024
I'm seeking a postdoc to work with me and @kenholstein.bsky.social on evaluating AI/ML decision support for human experts: statmodeling.stat.columbia.edu/2024/12/10/p... P.S. I'll be at NeurIPS Thurs-Mon. Happy to talk about this position or related mutual interests! Please repost 🙏
statmodeling.stat.columbia.edu
Postdoc position at Northwestern on evaluating AI/ML decision support | Statistical Modeling, Causal Inference, and Social Science
03218
Reposted by Berk Ustun
Shakir Mohamed @shakirm.bsky.social · 11/12/2024
There is just about a month left before the abstract deadline for @facct.bsky.social 2025🥳😳🙈. Really looking forward to seeing all the submissions. If you are submitting, don’t forget to check out the Author Guide (and the reviewer and AC guide as well). facctconference.org/2025/aguide
facctconference.org
ACM FAccT - 2025 Home
0154
Reposted by Berk Ustun
BIFOLD Berlin Institute for the Foundations of Learning and Data @bifold.berlin · 11/12/2024
Starter pack #ML for Healthcare go.bsky.app/PJKJ8vK by ‪@berkustun.bsky.social‬
021
Reposted by Berk Ustun
Stephanie Hyland @hylandsl.bsky.social · 09/12/2024
My group is hiring postdoc(s) and I will be at #NeurIPS2024 if you want to talk - send me a DM or find me on Whova! UK-based, 2 year position. We work on AI/ML for health, specifically building and *understanding/explaining* deep (multimodal) models on healthcare data, especially medical imaging.
0115
Reposted by Berk Ustun
Alondra Nelson @alondra.bsky.social · 06/12/2024
“Denied by AI,” the multi-part STAT News investigation of how #UnitedHealthcare used an opaque algorithmic system to deny care to people who needed it is a #mustread www.statnews.com/2023/03/13/m...
statnews.com
Denied by AI: How Medicare Advantage plans use algorithms to cut off care for seniors in need
A STAT investigation found artificial intelligence is driving Medicare Advantage denials to new heights, cutting off care for seniors.
251135543
Reposted by Berk Ustun
Konrad Kording @kordinglab.bsky.social · 03/12/2024
Personally I think they should have a superset of: (1) empower their students (2) foment collaborations (3) help produce clarity (4) intellectually move to their students not other way around (5) teach core skills (6) mentor effectively (7) produce a sense of safety and joy
2275
Reposted by Berk Ustun
Ben Recht @beenwrekt.bsky.social · 27/11/2024
This is a great thread that lists many of the features: bsky.app/profile/scie...
052
Reposted by Berk Ustun
Jesse Engreitz @jengreitz.bsky.social · 25/11/2024
Introducing scE2G: a new model to link enhancers to target genes using single-cell data. Excited that scE2G will enable building enhancer maps in hundreds of cell types in the human body! Wonderful collaboration with @randersson.bsky.social @613weilin.bsky.social @mayayayas.bsky.social Thread👇
0298
Reposted by Berk Ustun
Julian Skirzynski @jskirzynski.bsky.social · 23/11/2024
I tried to find everyone who works in the area but I certainly missed some folks so please lmk... go.bsky.app/BYkRryU
325318
Reposted by Berk Ustun
Berk Ustun @berkustun.bsky.social · 17/11/2024
Couldn't find a machine learning for health starter pack so I made one.  DM/Reply if you want to be added! go.bsky.app/PJKJ8vK
4810929
Reposted by Berk Ustun
Santiago Viquez @santiviquez.com · 19/11/2024
I heard bluesky likes links. So here is a link to a book I’m writing. github.com/NannyML/The-...
github.com
GitHub - NannyML/The-Little-Book-of-ML-Metrics: The book every data scientist needs on their desk.
The book every data scientist needs on their desk. - NannyML/The-Little-Book-of-ML-Metrics
28315
Reposted by Berk Ustun
sorelle @friedler.net · 14/11/2024
Ok #FAccT folks - I'm still looking around for people. Who am I missing? go.bsky.app/EQF5Ne1
255818
Berk Ustun @berkustun.bsky.social · 17/11/2024
Couldn't find a machine learning for health starter pack so I made one.  DM/Reply if you want to be added! go.bsky.app/PJKJ8vK
4810929
Reposted by Berk Ustun
Uri Shalit @urish.bsky.social · 15/11/2024
There’s a long line of literature starting from the ICP paper arxiv.org/abs/1501.01332, with IRM being an important (if imperfect) checkpoint arxiv.org/abs/1907.02893 I also really like the anchor regression paper arxiv.org/abs/1801.06229 and this is our work (calibrate!) arxiv.org/abs/2102.10395
arxiv.org
Causal inference using invariant prediction: identification and confidence intervals
What is the difference of a prediction that is made with a causal model and a non-causal model? Suppose we intervene on the predictor variables or change the whole environment. The predictions from a ...
1122
Reposted by Berk Ustun
Jessica Hullman @jessicahullman.bsky.social · 11/09/2024
We (me & @berkustun.bsky.social) created a starter pack of researchers working on algorithmic decision-making & fairness. Other platforms still seem to dominate for many ML-related topics, but maybe this will help Probably missed people, so let us know if you have suggestions! go.bsky.app/851zVkg
76223
Berk Ustun @berkustun.bsky.social · 27/11/2023
Please share! We're hiring postdoctoral researchers to work on responsible machine learning at UCSD! Topics include fairness, explainability, robustness, and safety. For more, see berkustun.com/postdoc/
berkustun.com
Berk Ustun
010