Ana Marasović @anamarasovic.bsky.social · 19/06/2026One more CoLM meta-review to write, yay... ...the last paper's reviews+discussion contain about **16,000 words** and the paper is a borderline, RIP 040
Ana Marasović @anamarasovic.bsky.social · 16/06/2026How uninformed you have to be to suggest this prompt... “Create a research grant proposal focusing on [topic]. Align it with [funding agency] guidelines (linked) and ensure the proposal includes a robust literature review, detailed methods section [...]” academy.openai.com/public/clubs...academy.openai.comPrompt Pack for Faculty - Resource | OpenAI AcademyReady-to-use prompt examples to maximize the benefits of ChatGPT Edu in teaching and research. 382
Ana Marasović @anamarasovic.bsky.social · 06/05/2026Why is 4 still the minimum number of papers one can sign up to review for ARR?? 150
Ana Marasović @anamarasovic.bsky.social · 19/03/2026Is it just me or Claude chat interface is really buggy these days? 020
Reposted by Ana MarasovićEugene Vinitsky 🍒 @eugenevinitsky.bsky.social · 15/03/2026For a recent lab meeting, I wrote up a grab bag of ways to think about your development as a researcher during a PhD: emerge-lab.github.io/papers/an-un... Sharing in case folks find it useful or have feedback!emerge-lab.github.io 610112
Reposted by Ana MarasovićKenneth Marino @kennethmarino.bsky.social · 04/03/2026Been less than a year since I started my lab at @utah.edu and we already have a ton of new stuff that I can’t wait to talk about soon. I’ll start today by sharing that our updated Computer Use Survey blog has been accepted to ICLR Blogposts 2026. iclr-blogposts.github.io/2026/blog/20... 163
Ana Marasović @anamarasovic.bsky.social · 04/03/2026Great test for anyone learning mech interp is reading Nikankin et al's "Arithmetic Without Algorithms" which uses activation patching / circuits, probing, logit lens, describing max activating examples.. If you you follow along while reading, you'll realize you know a lot! arxiv.org/abs/2410.21272arxiv.orgArithmetic Without Algorithms: Language Models Solve Math With a Bag of HeuristicsDo large language models (LLMs) solve reasoning tasks by learning robust generalizable algorithms, or do they memorize training data? To investigate this question, we use arithmetic reasoning as a rep... 1164
Ana Marasović @anamarasovic.bsky.social · 23/02/2026Getting sick in the middle of the semester, like catching a flu, makes every next week of the semester progressively worse. Will I ever catch up 😭 280
Reposted by Ana MarasovićNaomi Saphra @nsaphra.bsky.social · 03/02/2026I know grant writing is supposed to be miserable. Maybe once I've written a few and had them all rejected I will agree. But it's kind of my favorite genre to write currently? It's pure optimism! These are my hopes for the future! My favorite ideas! Untainted by messy and limited results! 6511
Ana Marasović @anamarasovic.bsky.social · 30/01/2026Somehow before becoming a prof I came away with the impression that grant writing was an annoying task profs have to do, and yes, more rejection sucks, but it is wonderful to start new collaborations with super smart people, brainstorm hard, and think on a larger scale than next few papers 1232
Ana Marasović @anamarasovic.bsky.social · 29/01/2026It's so depressing to see 20-30K papers submitted to every top ML conference 2130
Ana Marasović @anamarasovic.bsky.social · 12/01/2026Mathematicians are really nice to their LLMs; seen in: arxiv.org/abs/2601.01235 060
Ana Marasović @anamarasovic.bsky.social · 05/01/2026We need to return word limit to author responses 260
Reposted by Ana MarasovićTucker Hermans @thermans.bsky.social · 16/12/2025We are hiring in AI and NLP at Utah! Please apply if interested and share with folks you know are looking for faculty positions, especially if they like to ski, hike, climb, or bike! utah.peopleadmin.com/postings/190...utah.peopleadmin.comAssistant Professor for The Kahlert School of Computing 062
Reposted by Ana MarasovićScientific Computing & Imaging Institute @sci.utah.edu · 10/12/2025💫 On the heels of announcing 12 new faculty fellows last week, SCI's One-U Responsible AI Initiative is excited to add three new postdoctoral fellows to its team next year: rai.utah.edu/postdocs-dec...rai.utah.eduNew Postdocs Will Advance Tutor Chatbots, Bionic Hands, and Understanding of Pollution’s Impact on Reproductive Health - One-U Responsible AI InitiativeDecember 10, 2025 Mingjia Hu Mentors: One-U RAI faculty fellow Chenglu Li, assistant professor, Department of Educational Psychology; One-U RAI faculty fellow Ana Marasović, assistant... 041
Reposted by Ana MarasovićEric Eide @ericeide.bsky.social · 04/12/2025KSL TV @ksl.com interviewed the members of my research group who recently discovered a rare piece of computing history: an old tape that might contain UNIX V4. ksltv.com/science-tech...ksltv.comUniversity of Utah team discovers rare computer relicA research team at the University of Utah uncovered a rare piece of computing history. 073
Reposted by Ana MarasovićScientific Computing & Imaging Institute @sci.utah.edu · 02/12/2025📣 Meet the 12 new faculty fellows joining SCI’s One-U Responsible AI Initiative. These professors are advancing AI research to solve real-world challenges, from protecting the West’s water supply to improving medical care to embedding ethics in AI education. bit.ly/rai-fac-26bit.lyOne-U RAI Welcomes 12 New Faculty Fellows Driving Responsible AI Across Disciplines - One-U Responsible AI InitiativeDecember 1, 2025 The University of Utah One-U Responsible Artificial Intelligence Initiative (One-U RAI) at the Scientific Computing and Imaging (SCI) Institute has named 12... 072
Ana Marasović @anamarasovic.bsky.social · 20/11/2025Is it me or more reviewers are -strongly- opinionated about how you paper should be written? I feel like my responses in the past year or two are turning into arguments about why my writing is just fine 190
Ana Marasović @anamarasovic.bsky.social · 19/11/2025Current ChatGPT response format is such an eyesore, what’s up with all the lists and emojis 140
Ana Marasović @anamarasovic.bsky.social · 16/11/2025Luca Guadagnino making an OpenAI movie, what's happening 👀👀 100
Ana Marasović @anamarasovic.bsky.social · 14/11/2025I didn't submit to ICLR, but I'm pretty sure next week I'll see similar quality issues in ARR I feel like execs of major ML/AI conferences from ICLR/NeurIPS/ICML/AAAI, ACL/EMNLP, to CVPR should sit together and figure out a whole new strategy moving forward like 👇 3140
Ana Marasović @anamarasovic.bsky.social · 13/11/2025When your husband is also an academic, so when you can't get him on a phone you shoot an email and it works every time 😂💀 3200
Reposted by Ana MarasovićPranav A @pranav-nlp.bsky.social · 21/10/2025We're surveying researchers about name changes in academic publishing. If you've changed your name and dealt with updating publications, we want to hear your experience. Any reason counts: transition, marriage, cultural reasons, etc. forms.cloud.microsoft/e/E0XXBmZdEP 21623
Ana Marasović @anamarasovic.bsky.social · 11/11/2025@mclemcrew.bsky.social's CoLM spotlight is now available on YT! 🎵 youtu.be/w6LNmADnlNw?...youtu.beMixAssist: An Audio-Language Dataset for Co-Creative AI Assistance in Music MixingYouTube video by Conference on Language Modeling 111
Reposted by Ana MarasovićMartin Tutek @mtutek.bsky.social · 11/11/2025*Urgently* looking for emergency reviewers for the ARR October Interpretability track 🙏🙏 ReSkies much appreciated 1210
Reposted by Ana MarasovićEMNLP @emnlpmeeting.bsky.social · 07/11/2025Outstanding paper (5/7): "Measuring Chain of Thought Faithfulness by Unlearning Reasoning Steps" by Martin Tutek, Fateme Hashemi Chaleshtori, Ana Marasovic, and Yonatan Belinkov aclanthology.org/2025.emnlp-m... 6/naclanthology.orgMeasuring Chain of Thought Faithfulness by Unlearning Reasoning StepsMartin Tutek, Fateme Hashemi Chaleshtori, Ana Marasovic, Yonatan Belinkov. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. 2025. 1113
Ana Marasović @anamarasovic.bsky.social · 07/11/2025𝙒𝙚'𝙧𝙚 𝙝𝙞𝙧𝙞𝙣𝙜 𝙣𝙚𝙬 𝙛𝙖𝙘𝙪𝙡𝙩𝙮 𝙢𝙚𝙢𝙗𝙚𝙧𝙨! KSoC: utah.peopleadmin.com/postings/190... (AI broadly) Education + AI: - utah.peopleadmin.com/postings/189... - utah.peopleadmin.com/postings/190... Computer Vision: - utah.peopleadmin.com/postings/183... 11610
Reposted by Ana MarasovićEMNLP @emnlpmeeting.bsky.social · 07/11/2025🎉 Congratulations to all #EMNLP2025 award winners 🎉 Starting with the ✨Best Paper award ✨: "Infini-gram mini: Exact n-gram Search at the Internet Scale with FM-Index" by Hao Xu, Jiacheng Liu, Yejin Choi, Noah A. Smith, and Hannaneh Hajishirzi aclanthology.org/2025.emnlp-m... 1/n 1365
Ana Marasović @anamarasovic.bsky.social · 07/11/2025Thrilled to see this work recognized at #EMNLP2025! This framework and approach to measuring CoT faithfulness have been hugely influential for how I think about reasoning evaluation, and I'm so lucky to have worked with such brilliant collaborators. Huge credit to @mtutek.bsky.social 1150
Reposted by Ana MarasovićMartin Tutek @mtutek.bsky.social · 07/11/2025Very honored to be one out of seven outstanding papers at this years' EMNLP :) Huge thanks to my amazing collaborators @fatemehc.bsky.social @anamarasovic.bsky.social @boknilev.bsky.social , this would not have been possible without them! 2236
Ana Marasović @anamarasovic.bsky.social · 04/11/2025Check out Martin's talk at #EMNLP2025 today (Wed)! If you care about CoT faithfulness, you 𝘮𝘶𝘴𝘵 read this paper. It introduces the first method for measuring CoT faithfulness that is not purely behavioral, but operates with the internals! 070
Ana Marasović @anamarasovic.bsky.social · 04/11/2025Go check Alex's poster today (Wed) in Suzhou! #EMNLP2025 I'm still so proud of our work (led by @lasha.bsky.social) on CondaQA, so we had to ask what would happen if we tried to create high-quality reasoning-over-text benchmarks now that LLMs are available. Turns out, we'd make an easier benchmark! 081
Reposted by Ana MarasovićMartin Tutek @mtutek.bsky.social · 31/10/2025Flying out to @emnlpmeeting soon🇨🇳 I'll present our parametric CoT faithfulness work (arxiv.org/abs/2502.14829) on Wednesday at the second Interpretability session, 16:30-18:00 local time A104-105 If you're in Suzhou, reach out to talk all things reasoning :)arxiv.orgMeasuring Chain of Thought Faithfulness by Unlearning Reasoning StepsWhen prompted to think step-by-step, language models (LMs) produce a chain of thought (CoT), a sequence of reasoning steps that the model supposedly used to produce its prediction. Despite much work o... 0112
Reposted by Ana MarasovićEugene Vinitsky 🍒 @eugenevinitsky.bsky.social · 03/11/2025Sort of starting to believe that we really do need academic metrics that punish publishing too much 47810
Reposted by Ana MarasovićAlex Gill @agill32.bsky.social · 03/11/2025I'll be in Suzhou 🇨🇳 at #EMNLP this week presenting "What has been Lost with Synthetic Evaluation?" done with @anamarasovic.bsky.social & @lasha.bsky.social! 🎉 📍Findings Session 1 - Hall C 📅 Wed, November 5, 13:00 - 14:00 arxiv.org/abs/2505.22830 0112
Reposted by Ana MarasovićWomen in AI Research - WiAIR @wiair.bsky.social · 20/10/2025🧠 Can large language models build the very benchmarks used to evaluate them? In “What Has Been Lost with Synthetic Evaluation”, Ana Marasović (@anamarasovic.bsky.social) and collaborators ask what happens when LLMs start generating the datasets used to test their reasoning. (1/6🧵) 293
Reposted by Ana MarasovićWomen in AI Research - WiAIR @wiair.bsky.social · 15/10/2025👉 Do large language models really reason the way their chain-of-thoughts suggest? This week on #WiAIRpodcast, we talk with Ana Marasović (@anamarasovic.bsky.social) about her paper “Chain-of-Thought Unfaithfulness as Disguised Accuracy.” (1/6🧵) 📄 Paper: arxiv.org/pdf/2402.14897 151
Reposted by Ana MarasovićDaniel Brown @daniel-brown.bsky.social · 10/10/2025Can you trust your reward model alignment scores? New work presented today at the COLM Workshop on Socially Responsible Language Modelling Research led by Purbid Bambroo and in collaboration with @anamarasovic.bsky.social that probes LLM preference test sets for redundancy and inflated scores. 1/8 121
Reposted by Ana MarasovićAna Marasović @anamarasovic.bsky.social · 09/10/2025📣Tomorrow at #COLM2025: 1️⃣ Purbid's 𝐩𝐨𝐬𝐭𝐞𝐫 at 𝐒𝐨𝐋𝐚𝐑 (𝟏𝟏:𝟏𝟓𝐚𝐦-𝟏:𝟎𝟎𝐩𝐦) on catching redundant preference pairs & how pruning them hurts accuracy; www.anamarasovic.com/publications... 2️⃣ My 𝐭𝐚𝐥𝐤 at 𝐗𝐋𝐋𝐌-𝐑𝐞𝐚𝐬𝐨𝐧-𝐏𝐥𝐚𝐧 (𝟏𝟐𝐩𝐦) on measuring CoT faithfulness by looking at internals, not just behaviorally 1/3 1143
Ana Marasović @anamarasovic.bsky.social · 09/10/2025📣Tomorrow at #COLM2025: 1️⃣ Purbid's 𝐩𝐨𝐬𝐭𝐞𝐫 at 𝐒𝐨𝐋𝐚𝐑 (𝟏𝟏:𝟏𝟓𝐚𝐦-𝟏:𝟎𝟎𝐩𝐦) on catching redundant preference pairs & how pruning them hurts accuracy; www.anamarasovic.com/publications... 2️⃣ My 𝐭𝐚𝐥𝐤 at 𝐗𝐋𝐋𝐌-𝐑𝐞𝐚𝐬𝐨𝐧-𝐏𝐥𝐚𝐧 (𝟏𝟐𝐩𝐦) on measuring CoT faithfulness by looking at internals, not just behaviorally 1/3 1143
Ana Marasović @anamarasovic.bsky.social · 09/10/2025Sad: Can't go to CoLM because of immigration. Happy: Well, at least I can mountain bike during the fall break in prime SLC MTB weather. Sad: Comes down with a cold. ☹️☹️☹️☹️☹️☹️ 070
Ana Marasović @anamarasovic.bsky.social · 08/10/2025I had a great time chatting with Jekaterina and Malikeh. This episode is like a tour of all the things I've been studying lately! 010
Reposted by Ana MarasovićWomen in AI Research - WiAIR @wiair.bsky.social · 08/10/2025🎙️ New Women in AI Research episode out now! This time, we sit down with @anamarasovic.bsky.social to unpack some of the toughest questions in AI explainability and trust. 🔗 Watch here → youtu.be/xYb6uokKKOoyoutu.be 111
Reposted by Ana MarasovićYonatan Belinkov @boknilev.bsky.social · 07/10/2025In #Interplay25 workshop, Friday ~11:30, I'll present on measuring *parametric* CoT faithfulness on behalf of @mtutek.bsky.social , who couldn't travel: bsky.app/profile/mtut... Later that day we'll have a poster on predicting success of model editing by Yanay Soker, who also couldn't travelx.comMartin Tutek on X: "🚨🚨 New preprint 🚨🚨 Ever wonder whether CoTs correspond to the internal reasoning process of the model? We propose a novel parametric faithfulness approach, which erases information contained in CoT steps from parameters to assess CoT faithfulness. https://t.co/WZDUZbJbxC" / X🚨🚨 New preprint 🚨🚨 Ever wonder whether CoTs correspond to the internal reasoning process of the model? We propose a novel parametric faithfulness approach, which erases information contained in CoT steps from parameters to assess CoT faithfulness. https://t.co/WZDUZbJbxC 141
