Sign in

Vidhisha Balachandran

@vidhishab.bsky.social
712 followers 107 following 7 posts

AI Evaluation and Interpretability @MicrosoftResearch, Prev PhD @CMU.

PostsRepliesMedia
Vidhisha Balachandran @vidhishab.bsky.social · 27/05/2025
All our evaluation logs and reasoning traces for open source models are now released! We hope this can be useful for the community for further research and analysis!
000
Reposted by Vidhisha Balachandran
Besmira Nushi @besmiranushi.bsky.social · 29/04/2025
All Eureka inference-time scaling insights are now available here: www.microsoft.com/en-us/resear... It was fun sharing these and more together with Vidhisha Balachandran @vidhishab.bsky.social and Vibhav Vineet at #ICLR2025.
microsoft.com
Eureka Inference-Time Scaling Insights: Where We Stand and What Lies Ahead - Microsoft Research
Understanding and measuring the potential of inference-time scaling for reasoning. The new Eureka study tests nine state-of-the-art models on eight diverse reasoning tasks.
032
Reposted by Vidhisha Balachandran
Besmira Nushi @besmiranushi.bsky.social · 01/05/2025
🎉The Phi-4 reasoning models have landed on HF and Azure AI Foundry. The new models are competitive and often outperform much larger frontier models. It is exciting to see the reasoning capabilities extend to more domains beyond math, including algorithmic reasoning, calendar planning, and coding.
1208
Reposted by Vidhisha Balachandran
Besmira Nushi @besmiranushi.bsky.social · 24/04/2025
Come see us in any of the following sessions on model understanding and evaluation! 🔬 #ICLR2025 @msftresearch.bsky.social
011
Reposted by Vidhisha Balachandran
Alessandro Stolfo @alestolfo.bsky.social · 15/04/2025
Our paper "Improving Instruction-Following in Language Models through Activation Steering” has been accepted to #ICLR2025! We're also excited to share that our public GitHub repo is now live. Code: github.com/microsoft/ll... Camera-ready: arxiv.org/abs/2410.12877
182
Vidhisha Balachandran @vidhishab.bsky.social · 10/04/2025
🚀 Excited to share our new Eureka report! We studied inference-time scaling across 9 models (conventional & reasoning) on 8 tough tasks—from math & STEM reasoning to navigation, calendar planning, NP-hard problems & spatial planning. Full Report: aka.ms/eureka-ml-in...
aka.ms
Inference-Time Scaling for Complex Tasks: Where We Stand and What Lies Ahead - Microsoft Research
This work investigates the benefits and limitations of inference-time scaling methods across nine state-of-the-art models and eight challenging tasks, including math and STEM reasoning, calendar plann...
010
Reposted by Vidhisha Balachandran
Stella Li @stellali.bsky.social · 21/02/2025
Asking the right questions can make or break decisions in fields like medicine, law, and beyond✴️ Our new framework ALFA—ALignment with Fine-grained Attributes—teaches LLMs to PROACTIVE seek information through better questions through **structured rewards**🏥❓ (co-led with @jiminmun.bsky.social) 👉🏻🧵
1247
Reposted by Vidhisha Balachandran
Tsvetshop NLP @tsvetshop.bsky.social · 24/02/2025
Effective decision-making starts with asking the right questions. Our new framework, ALFA, teaches LLMs to ask questions through fine-grained attributes in expert domains. Excited to see where this takes the next generation of effective LLM assistants and agents!
021
Vidhisha Balachandran @vidhishab.bsky.social · 15/12/2024
Excited to share our December updates on the state of progress in AI ! @msftresearch.bsky.social Detailed report coming early next year ✨
080
Reposted by Vidhisha Balachandran
dheerajr.bsky.social @dheerajr.bsky.social · 14/12/2024
Stoked to share our new work on scaling training data attribution (TDA) toward LLM pretraining - and great insights we found along the way! medium.com/people-ai-re... and more in the thread below from most excellent student researcher @tylerachang.bsky.social
medium.com
Scaling Training Data Attribution
The science of how training data influences LLM behavior
0121
Reposted by Vidhisha Balachandran
Shital Shah @sytelus.bsky.social · 13/12/2024
Are you ready for an early Christmas present from our team at Microsoft Research? Introducing the most powerful smol model ever built in the world! Welcome to Phi-4! 👇
1121
Reposted by Vidhisha Balachandran
Besmira Nushi @besmiranushi.bsky.social · 13/12/2024
The phi-4 technical report is now available on arxiv arxiv.org/abs/2412.08905 and on Azure AI. Congratulations to the phi team on the release and the major milestone on scaling data quality processes! 🎉 @msftresearch.bsky.social @sbubeck.bsky.social @suriyag.bsky.social @sytelus.bsky.social
041
Vidhisha Balachandran @vidhishab.bsky.social · 12/12/2024
Come talk to us about model evaluation! 4:30 pm today at West Meeting Room 301 Also to see @besmiranushi.bsky.social ‘s cool demos 🍁
081
Vidhisha Balachandran @vidhishab.bsky.social · 09/12/2024
We will be presenting Eureka - our model evaluation framework and sharing in-depth insights at NeurIPS this week! Come join us on Wednesday (Dec 11) 4:30pm at West Meeting Room 301 to hear what we’ve been upto the past few months! :) neurips.cc/Expo/Confere... microsoft.github.io/eureka-ml-in...
040
Reposted by Vidhisha Balachandran
Anka Reuel ➡️ NeurIPS @ankareuel.bsky.social · 25/11/2024
🚨 NeurIPS 2024 Spotlight Did you know we lack standards for AI benchmarks, despite their role in tracking progress, comparing models, and shaping policy? 🤯 Enter BetterBench–our framework with 46 criteria to assess benchmark quality: betterbench.stanford.edu 1/x
413925
Reposted by Vidhisha Balachandran
Besmira Nushi @besmiranushi.bsky.social · 26/11/2024
Today is the International Day for the Elimination of Violence against Women. According to the UN, more than 50 000 women were killed by a partner or family member in 2023 news.un.org/en/story/202... This number is an underestimate given that only 37 countries reported in 2023.
083