Sign in

Alexandra Olteanu

@aolteanu.bsky.social
1.8K followers 498 following 88 posts

Ethical/Responsible AI. Rigor in AI. Grumpy eastern european in north america. Lovingly nitpicky. www.aolteanu.com

PostsRepliesMedia
Alexandra Olteanu @aolteanu.bsky.social · 08/08/2026
As we have discussions about e.g., “authorship quotas” for conferences and “authorship when using LLMs” it may also be worth thinking about how each of us approaches what authorship should entail. A few thoughts about how I think about this:
scholarly-things.leaflet.pub
On (co-)authorship
TL;DR — Authorship requires both earning it and being able to defend the work.
051
Alexandra Olteanu @aolteanu.bsky.social · 02/08/2026
So what do AI evaluations do or should do? Wrote a brief reflection about prototypical questions evaluations often try to answer and why it is useful to have clarity about what exactly one is trying to learn about some phenomenon of interest. rigor-in-ai.leaflet.pub/3ms4ax4sojs2j
rigor-in-ai.leaflet.pub
Back-to-basics: so what do evaluations do
TL;DR — Developing useful evaluations requires clarity about what exactly one is trying to learn about a phenomenon of interest.
092
Alexandra Olteanu @aolteanu.bsky.social · 25/07/2026
Wrote another brief post motivated by a reflection on how in AI research folks might not fully appreciate the care that working with unobservable constructs requires. TL;DR -- Because unobservable constructs hold ‘surplus meaning,’ your metric is not your construct.
rigor-in-ai.leaflet.pub
Back-to-basics: unobservable constructs and their ‘surplus meaning’
TL;DR — Because unobservable constructs hold ‘surplus meaning,’ your metric is not your construct.
1173
Alexandra Olteanu @aolteanu.bsky.social · 20/07/2026
Wrote a brief reflection on poor conceptualizations in AI work and why they matter, something I believe is so startlingly neglected. TL;DR - Poor conceptual foundations can severely undermine the credibility and reliability of knowledge claims.
rigor-in-ai.leaflet.pub
Back-to-basics: on poor conceptualizations in AI work
TL;DR — Poor conceptual foundations can severely undermine the credibility and reliability of knowledge claims. (And, no, your metric is not your construct.)
2105
Alexandra Olteanu @aolteanu.bsky.social · 15/07/2026
I'm thinking about writing brief reflections on a few topics (mainly from my own attempts to gain clarity about research practices I find concerning). I'm thinking about using medium, but I see many folks use substack. Are there other platforms? What's your reasoning for using a certain platform?
250
Reposted by Alexandra Olteanu
Emily M. Bender @emilymbender.bsky.social · 07/07/2026
I have a foolproof way to make sure that your papers contain absolutely no fake references . . . . . read the papers you cite.
201038216
Reposted by Alexandra Olteanu
minaremeli.bsky.social @minaremeli.bsky.social · 08/07/2026
Come meet me at the last poster session at ICML if you are interested in chatting about LLM evaluation and pairwise comparisons! I will be presenting joint work with my advisor (Moritz Hardt). Thu Jul 9, 5:00 PM – 6:45 PM KST, Hall A (#4411) arxiv.org/abs/2606.09409
arxiv.org
Correct Looks Better: Pairwise Comparisons Reveal Accuracy Rankings
Pairwise comparisons combined with aggregation methods like Elo have become central to evaluating generative models, yet concerns remain that they reward superficial stylistic cues or display judge bi...
1132
Alexandra Olteanu @aolteanu.bsky.social · 20/06/2026
So often folks conflate outputs with outcomes; in most settings, the system outputs are different than the outcomes of interest.
020
Alexandra Olteanu @aolteanu.bsky.social · 30/05/2026
Sometimes it feels like some folks are losing the plot about what the goal of publishing research actually is. The goal is certainly not meant to be the productions of papers, but rather the production and communication of science.
1173
Alexandra Olteanu @aolteanu.bsky.social · 03/03/2026
Yesterday was my last day at MSR. We recently learned that our roles were eliminated, and with them our little FATE Montreal team. I joined MSR a bit over 7.5 years ago while on active chemotherapy, and being at MSR has overlapped with so much change in my life.
4346
Reposted by Alexandra Olteanu
Alexandra Olteanu @aolteanu.bsky.social · 18/02/2026
The deadline for the 2026 FAccT DC is next Tuesday, February 24! If you are a student working on topics relevant to the FAccT's scope, this is an opportunity to interact with a diverse set of peers and mentors! #facct2026 #facct26 #facct Details here: facctconference.org/2026/callfor...
facctconference.org
ACM FAccT 2026 Doctoral Colloquium Call for Applications
0810
Alexandra Olteanu @aolteanu.bsky.social · 18/02/2026
The deadline for the 2026 FAccT DC is next Tuesday, February 24! If you are a student working on topics relevant to the FAccT's scope, this is an opportunity to interact with a diverse set of peers and mentors! #facct2026 #facct26 #facct Details here: facctconference.org/2026/callfor...
facctconference.org
ACM FAccT 2026 Doctoral Colloquium Call for Applications
0810
Alexandra Olteanu @aolteanu.bsky.social · 21/01/2026
We are hoping to see applications from and bring together a diverse group of students across multiple disciplinary areas! If you are a graduate student and interested in FAccT’s scope, the Doctoral Colloquium is for you! #facct2026 #facct26
053
Reposted by Alexandra Olteanu
Alexandra Olteanu @aolteanu.bsky.social · 05/12/2025
📣 It's again that time of the year 🤩 - our internship call for the FATE team @ MSR Montreal and our collaborators is now up! 🎉🎉 We are broadly looking for candidates interested in perceptions, evaluation, uses, and impacts of AI! #FAccT #FAccT2025 #CHI2025 #NeurIPS2025 #ACL2025
1259
Alexandra Olteanu @aolteanu.bsky.social · 05/12/2025
📣 It's again that time of the year 🤩 - our internship call for the FATE team @ MSR Montreal and our collaborators is now up! 🎉🎉 We are broadly looking for candidates interested in perceptions, evaluation, uses, and impacts of AI! #FAccT #FAccT2025 #CHI2025 #NeurIPS2025 #ACL2025
1259
Alexandra Olteanu @aolteanu.bsky.social · 03/12/2025
@zey.bsky.social's paper and talk at ICWSM'14 (my first ICWSM!) shaped my thinking on issues around data/technology and the type of problems I still spend my time working on/thinking about. Can't think of a better keynote speaker for my first NeurIPS.
110
Alexandra Olteanu @aolteanu.bsky.social · 01/12/2025
On my way to @neuripsconf.bsky.social -- this is my first time at #NeurIPS and looking forward to seeing folks and presenting our paper on Rigor in AI. Do find me if you want to chat about rigor in AI, anthropomorphic AI, or evaluations #NeurIPS2025
Screenshot of the first page of the paper titled "rigor in ai: doing rigorous AI work requires a broader, responsible AI-informed conception of rigor"
0162
Reposted by Alexandra Olteanu
Lucy Li @lucy3.bsky.social · 11/11/2025
It's the season for PhD apps!! 🥧 🦃 ☃️ ❄️ Apply to Wisconsin CS to research - Societal impact of AI - NLP ←→ CSS and cultural analytics - Computational sociolinguistics - Human-AI interaction - Culturally competent and inclusive NLP with me! lucy3.github.io/prospective-...
A staircase in the new School of Computer, Data & Information Sciences building at Wisconsin Madison. Tan wood structures surround tapestry art and a small indoor garden.A view from above of the staircases in the Wisconsin CDIS building An shot from below of winding wooden staircases and a glass atrium rooftop. The new School of Computer, Data & Information Sciences building at Wisconsin Madison. A bicolor white cat with seal-colored markings, looking upwards with big wide dark eyes.
15116
Alexandra Olteanu @aolteanu.bsky.social · 11/11/2025
Unexpected (amount of) snow day
Snow on the balcony
090
Reposted by Alexandra Olteanu
Michael Ekstrand @md.ekstrandom.net · 07/11/2025
Our forthcoming NeurIPS position paper, led by @aolteanu.bsky.social, makes this argument (along with several related ones) in more depth. Rigorous AI/ML work should flow from explicit and rigorous premises, not just have a final evaluation that checks some rigor boxes. arxiv.org/abs/2506.14652
arxiv.org
Rigor in AI: Doing Rigorous AI Work Requires a Broader, Responsible AI-Informed Conception of Rigor
In AI research and practice, rigor remains largely understood in terms of methodological rigor -- such as whether mathematical, statistical, or computational methods are correctly applied. We argue th...
152
Alexandra Olteanu @aolteanu.bsky.social · 18/10/2025
I wish folks would use more precise terminology than "AI sycophancy." Not all validating behaviours/interactions are sycophantic. By definition, for them to be sycophantic there needs to be an underlying intention to e.g., gain advantage or favour. Intention is something AI systems do not have.
3110
Alexandra Olteanu @aolteanu.bsky.social · 01/10/2025
Love this analogy
1294
Alexandra Olteanu @aolteanu.bsky.social · 30/09/2025
This was accepted to #NeurIPS 🎉🎊 TL;DR Impoverished notions of rigor can have a formative impact on AI work. We argue for a broader conception of what rigorous work should entail & go beyond methodological issues to include epistemic, normative, conceptual, reporting & interpretative considerations
1268
Reposted by Alexandra Olteanu
Dr. Chinasa T. Okolo @chinasa.bsky.social · 12/09/2025
"Epistemic rigor, however, does not necessarily require specific epistemological commitments or choices but rather that those commitments and choices be made explicit." www.arxiv.org/abs/2506.14652
arxiv.org
Rigor in AI: Doing Rigorous AI Work Requires a Broader, Responsible AI-Informed Conception of Rigor
In AI research and practice, rigor remains largely understood in terms of methodological rigor -- such as whether mathematical, statistical, or computational methods are correctly applied. We argue th...
182
Alexandra Olteanu @aolteanu.bsky.social · 31/07/2025
Listening to a workshop panel at #acl2025 I am realizing that we are saying more or less the same things and having more or less the same conversations for so many years
050
Alexandra Olteanu @aolteanu.bsky.social · 29/07/2025
#acl2025 I think there is plenty of evidence for the risks of anthropomorphic AI behavior and design (re: keynote) -- find @myra.bsky.social and I if you want to chat more about this or our "Dehumanizing Machines" ACL 2025 paper
0111
Reposted by Alexandra Olteanu
Melanie Mitchell @melaniemitchell.bsky.social · 21/07/2025
In a stunning moment of self-delusion, the Wall Street Journal headline writers admitted that they don't know how LLM chatbots work.
432936468
Alexandra Olteanu @aolteanu.bsky.social · 19/07/2025
Who is attending @aclmeeting.bsky.social in Vienna? Reach out or find me there if you want to chat! #acl2025nlp
050
Reposted by Alexandra Olteanu
Luke Stark @lukestark.bsky.social · 17/07/2025
My university has announced a fund to essentially poach doctoral students from US institutions. DM me if you do work on the history/social impacts of AI and are interested in being poached 😂
16590277
Alexandra Olteanu @aolteanu.bsky.social · 16/07/2025
Not sure who needs to hear this but what people want AI systems to do, what AI systems do, and what people believe AI systems do are not the same thing. Just because one wants or believes AI systems do or can do certain things, doesn't mean they actually do those things.
093
Reposted by Alexandra Olteanu
Hanna Wallach @hannawallach.bsky.social · 15/07/2025
If you're at @icmlconf.bsky.social this week, come check out our poster on "Position: Evaluating Generative AI Systems Is a Social Science Measurement Challenge" presented by the amazing @afedercooper.bsky.social from 11:30am--1:30pm PDT on Weds!!! icml.cc/virtual/2025...
icml.cc
ICML Poster Position: Evaluating Generative AI Systems Is a Social Science Measurement ChallengeICML 2025
13210
Reposted by Alexandra Olteanu
David Rothschild @davmicrot.bsky.social · 10/07/2025
Do you have strong programming skills but need research experience doing meaningful & exciting CSS projects before heading off to a top graduate school for computational social science PhD? Apply now to predoc with me, @dggoldst.bsky.social @jakehofman.bsky.social www.microsoft.com/en-us/resear...
microsoft.com
Predoctoral Research Assistant (Contract) – Computational Social Science - Microsoft Research
Are you a recent college graduate wishing to gain research experience prior to pursuing a Ph.D. in fields related to computational social science (CSS)? Do you have a deep love of “playing with data”—...
2109
Reposted by Alexandra Olteanu
Maria Antoniak @mariaa.bsky.social · 04/07/2025
Someone asked me today how to get better at scientific writing. I'm not the best person to ask because I find my own writing very inadequate! But the tips I thought of were: 1. Practice, and practice with co-authors who are better writers than you. Observe how they make edits and copy them. (1/n)
15611
Alexandra Olteanu @aolteanu.bsky.social · 27/06/2025
FAccT is such a special community & many of us have invested a lot of service time/effort to support it over the years. I do believe engaging with uncomfortable questions & dialogue is important even when there is criticism (which can be hard to hear, can feel unfair/demotivating & sucks) #facct2025
162
Reposted by Alexandra Olteanu
Haley L. @haleyhaala.bsky.social · 21/06/2025
Flattered and shocked for our paper to receive the #facct2025 best paper award.
1103
Alexandra Olteanu @aolteanu.bsky.social · 25/06/2025
Two years after the craft session on theories of change in responsible AI, I am glad to see this discussion taking central stage as a keynote panel #facct2025
0112
Alexandra Olteanu @aolteanu.bsky.social · 23/06/2025
There is a lot of talk and effort to figure out how genAI is different (I am also guilty of this!) -- the reality is that genAI is not that different and genAI is not that new either; it was hard to evaluate in the past, and it is still as hard to evaluate now #facct2025
0164
Reposted by Alexandra Olteanu
Asia Biega @asiabiega.bsky.social · 22/06/2025
Your #FAccT2025 General Chairs @sciorestis.bsky.social, @metaxa.net, and I, reporting from the venue. We're looking forward to welcoming you to the Athens Conservatoire or online!
1437
Alexandra Olteanu @aolteanu.bsky.social · 21/06/2025
Who is going to @facct.bsky.social? I will be arriving in Athens tomorrow late morning and looking forward to catching up with old and new friends at FAccT ☀️
3181
Alexandra Olteanu @aolteanu.bsky.social · 18/06/2025
We have to talk about rigor in AI work and what it should entail. The reality is that impoverished notions of rigor do not only lead to some one-off undesirable outcomes but can have a deeply formative impact on the scientific integrity and quality of both AI research and practice 1/
Print screen of the first page of a paper pre-print titled "Rigor in AI: Doing Rigorous AI Work Requires a Broader, Responsible AI-Informed Conception of Rigor" by Olteanu et al.  Paper abstract: "In AI research and practice, rigor remains largely understood in terms of methodological rigor -- such as whether mathematical, statistical, or computational methods are correctly applied. We argue that this narrow conception of rigor has contributed to the concerns raised by the responsible AI community, including overblown claims about AI capabilities. Our position is that a broader conception of what rigorous AI research and practice should entail is needed. We believe such a conception -- in addition to a more expansive understanding of (1) methodological rigor -- should include aspects related to (2) what background knowledge informs what to work on (epistemic rigor); (3) how disciplinary, community, or personal norms, standards, or beliefs influence the work (normative rigor); (4) how clearly articulated the theoretical constructs under use are (conceptual rigor); (5) what is reported and how (reporting rigor); and (6) how well-supported the inferences from existing evidence are (interpretative rigor). In doing so, we also aim to provide useful language and a framework for much-needed dialogue about the AI community's work by researchers, policymakers, journalists, and other stakeholders."
26518
Reposted by Alexandra Olteanu
Hanna Wallach @hannawallach.bsky.social · 15/06/2025
Alright, people, let's be honest: GenAI systems are everywhere, and figuring out whether they're any good is a total mess. Should we use them? Where? How? Do they need a total overhaul? (1/6)
13311
Reposted by Alexandra Olteanu
Hanna Wallach @hannawallach.bsky.social · 10/06/2025
I'm so excited this paper is finally online!!! 🎉 We had so much fun working on this with @emmharv.bsky.social!!! Thread below summarizing our contributions...
093
Reposted by Alexandra Olteanu
Emma Harvey @emmharv.bsky.social · 09/06/2025
📣 "Understanding and Meeting Practitioner Needs When Measuring Representational Harms Caused by LLM-Based Systems" is forthcoming at #ACL2025NLP - and you can read it now on arXiv! 🔗: arxiv.org/pdf/2506.04482 🧵: ⬇️
A screenshot of our paper: 

Title: Understanding and Meeting Practitioner Needs When Measuring Representational Harms Caused by LLM-Based Systems

Authors: Emma Harvey, Emily Sheng, Su Lin Blodgett, Alexandra Chouldechova, Jean Garcia-Gathright, Alexandra Olteanu, Hanna Wallach

Abstract: The NLP research community has made publicly available numerous instruments for measuring representational harms caused by large language model (LLM)-based systems. These instruments have taken the form of datasets, metrics, tools, and more. In this paper, we examine the extent to which such instruments meet the needs of practitioners tasked with evaluating LLM-based systems. Via semi-structured interviews with 12 such practitioners, we find that practitioners are often unable to use publicly available instruments for measuring representational harms. We identify two types of challenges. In some cases, instruments are not useful because they do not meaningfully measure what practitioners seek to measure or are otherwise misaligned with practitioner needs. In other cases, instruments---even useful instruments---are not used by practitioners due to practical and institutional barriers impeding their uptake. Drawing on measurement theory and pragmatic measurement, we provide recommendations for addressing these challenges to better meet practitioner needs.
1174
Reposted by Alexandra Olteanu
Hanna Wallach @hannawallach.bsky.social · 03/05/2025
Exciting news!!! This just got into @icmlconf.bsky.social as a position paper!!! 🎉 More updates to come as we work on the camera-ready version!!!
04911
Reposted by Alexandra Olteanu
Lucy Li @lucy3.bsky.social · 05/05/2025
I'm joining Wisconsin CS as an assistant professor in fall 2026!! There, I'll continue working on language models, computational social science, & responsible AI. 🌲🧀🚣🏻‍♀️ Apply to be my PhD student! Before then, I'll postdoc for a year in the NLP group at another UW 🏔️ in the Pacific Northwest
Wisconsin-Madison's tree-filled campus, next to a big shiny lake A computer render of the interior of the new computer science, information science, and statistics building. A staircase crosses an open atrium with visibility across multiple floors
1614514
Reposted by Alexandra Olteanu
Kaitlyn Zhou @kaitlynzhou.bsky.social · 24/04/2025
Life update! Excited to announce that I’ll be starting as an assistant professor at Cornell Info Sci in August 2026! I’ll be recruiting students this upcoming cycle! An abundance of thanks to all my mentors and friends who helped make this possible!!
19768
Reposted by Alexandra Olteanu
Myra Cheng @myra.bsky.social · 27/04/2025
This draws on work from last summer at MSR MTL, with @aolteanu.bsky.social, Su Lin Blodgett, @uhleeeeeeeshuh.bsky.social, and @lisaegede.bsky.social! Check out the full post: iclr-blogposts.github.io/2025/blog/an...
iclr-blogposts.github.io
“I Am the One and Only, Your Cyber BFF”: Understanding the Impact of GenAI Requires Understanding the Impact of Anthropomorphic AI | ICLR Blogposts 2025
State-of-the-art generative AI (GenAI) systems are increasingly prone to anthropomorphic behaviors, i.e., to generating outputs that are perceived to be human-like. While this has led to scholars incr...
062
Reposted by Alexandra Olteanu
Myra Cheng @myra.bsky.social · 27/04/2025
New ICLR blogpost! 🎉 We argue that understanding the impact of anthropomorphic AI is critical to understanding the impact of AI.
1154
Reposted by Alexandra Olteanu
Alicia DeVrio @uhleeeeeeeshuh.bsky.social · 27/04/2025
Presenting this at #CHI2025 tomorrow, Monday, April 28 in the "Expressive Machines" (lol 🤷‍♀️) session at 4:44 p.m. in Annex Hall F206
1241