Reposted by Anka Reuel ➡️ NeurIPSCas (Stephen Casper) @scasper.bsky.social · 21/04/2025🚨New paper: Current reports on AI audits/evals often omit crucial details, and there are huge disparities between the thoroughness of different reports. Even technically rigorous evals can offer little useful insight if reported selectively or obscurely. Audit cards can help. 142
Reposted by Anka Reuel ➡️ NeurIPSStanford HAI @stanfordhai.bsky.social · 25/03/2025A recent Stanford paper reveals that many popular AI benchmarks are fundamentally flawed: They can be outdated, easily gamed, or inaccurate. Stanford HAI Graduate Fellow @ankareuel.bsky.social talks about how researchers are rethinking AI benchmarks: www.emergingtechbrew.com/stories/2025...emergingtechbrew.comSome researchers are rethinking how to measure AI intelligenceCurrent popular benchmarks are often inadequate or too easy to game, experts say. 1103
Anka Reuel ➡️ NeurIPS @ankareuel.bsky.social · 28/01/2025Hey Kabir! A lot of it is applicable for different types of evals, especially when it comes to reporting considerations. Would you mind sharing more infos here or via DM on the hackathon? Sounds like this would be a cool opportunity to extend the BetterBench work! 140
Anka Reuel ➡️ NeurIPS @ankareuel.bsky.social · 27/01/2025Submitting a benchmark to ICML? Check out our NeurIPS Spotlight paper BetterBench! We outline best practices for benchmark design, implementation & reporting to help shift community norms. Be part of the change! 🙌 + Add your benchmark to our database for visibility: betterbench.stanford.edu 1133
Anka Reuel ➡️ NeurIPS @ankareuel.bsky.social · 05/01/2025This is such a hard one :D And I think it extends beyond being patient with the students but also being patient with yourself knowing that you won't get everything perfect the first time around (or ever 🥲) 160
Anka Reuel ➡️ NeurIPS @ankareuel.bsky.social · 05/01/2025🔄 Sharing is caring! Help us reach as wide of an audience as possible by spreading the word. Your support is key in crafting an insightful, community-driven chapter and help key researchers in the field get their work promoted! Thank you! 🙏#StanfordHAI #AIIndex x/ 052
Anka Reuel ➡️ NeurIPS @ankareuel.bsky.social · 05/01/2025The AI Index is an initiative by @stanfordhai.bsky.social. The annual report showcases AI research to enable decision-makers to advance AI responsibly. Previous versions have been cited 300+ times; it's been featured in top media outlets like the @nytimes.com & the @financialtimes.com. 4/ 150
Anka Reuel ➡️ NeurIPS @ankareuel.bsky.social · 05/01/2025Our chapter will cover fairness & non-discrimination, transparency, explainability, data governance & privacy, security, societal impact, and more. Plus, a special subchapter on responsible AI agents! 🤖 3/ 100
Anka Reuel ➡️ NeurIPS @ankareuel.bsky.social · 05/01/2025We're seeking impactful AI ethics/safety research from 2024/2025 for inclusion in Stanford's 2025 #AIIndex. Submit your papers or nominate others’ work through our Google Form👇 forms.gle/Hgrzvsi9Yb2B... 2/forms.gle2025 Stanford AI Index – Responsible AI ChapterThank you for submitting research for consideration for the responsible AI chapter for this year's Stanford AI Index by Stanford HAI! The AI Index is an independent initiative at the Stanford Instit... 111
Anka Reuel ➡️ NeurIPS @ankareuel.bsky.social · 05/01/2025📢 Excited to share: I'm again leading the efforts for the Responsible AI chapter for Stanford's 2025 AI Index, curated by @stanfordhai.bsky.social. As last year, we're asking you to submit your favorite papers on the topic for consideration (including your own!) 🧵 1/ 1148
Anka Reuel ➡️ NeurIPS @ankareuel.bsky.social · 04/01/2025This is all awesome advice, thank you so much for sharing! This is an in-person course but we’ll make all lectures publicly available. 010
Anka Reuel ➡️ NeurIPS @ankareuel.bsky.social · 04/01/2025I‘m teaching my first own course starting next week (Intro to AI Governance at Stanford). Super proud but also nervous 🥹 Any advice from more seasoned instructors? 😬 #AcademicTwitter #AcademicChatter #TeachingTips #AcademicAdvice 2110
Reposted by Anka Reuel ➡️ NeurIPSStephan Geering @stephangeering.bsky.social · 24/12/2024The regular reminder of my starter packs full of amazing folks / accounts to follow. I am trying to keep them up to date but let me know if I missed you. 051
Anka Reuel ➡️ NeurIPS @ankareuel.bsky.social · 19/12/2024As one of the vice chairs of the EU GPAI Code of Practice process, I co-wrote the second draft which just went online – feedback is open until mid-January, please let me know your thoughts, especially on the internal governance section! digital-strategy.ec.europa.eu/en/library/s...digital-strategy.ec.europa.euSecond Draft of the General-Purpose AI Code of Practice published, written by independent expertsIndependent experts present the second draft of the General-Purpose AI Code of Practice, based on the feedback received on the first draft, published on 14 November 2024. 0145
Reposted by Anka Reuel ➡️ NeurIPSStanford HAI @stanfordhai.bsky.social · 11/12/2024In our latest brief, Stanford scholars present a novel assessment framework for evaluating the quality of AI benchmarks and share best practices for minimum quality assurance. @ankareuel.bsky.social @chansmi.bsky.social @mlamparth.bsky.social hai.stanford.edu/what-makes-g... 0124
Anka Reuel ➡️ NeurIPS @ankareuel.bsky.social · 06/12/2024Thanks so much! And yes, very much looking forward to the weekend 😁🫶 010
Anka Reuel ➡️ NeurIPS @ankareuel.bsky.social · 06/12/2024In the same boat as @mlamparth.bsky.social, would appreciate if you could add me, too, please! Thanks so much 😊 220
Anka Reuel ➡️ NeurIPS @ankareuel.bsky.social · 06/12/2024In the same boat as @mlamparth.bsky.social, would appreciate if you could add me, too, please! Thanks so much 😊 120
Anka Reuel ➡️ NeurIPS @ankareuel.bsky.social · 06/12/2024In the same boat as @mlamparth.bsky.social, would appreciate if you could add me, too! Thanks so much 😊 120
Anka Reuel ➡️ NeurIPS @ankareuel.bsky.social · 06/12/2024In the same boat as @mlamparth.bsky.social, would appreciate if you could add me, too! Thanks so much 😊 020
Anka Reuel ➡️ NeurIPS @ankareuel.bsky.social · 06/12/2024Would appreciate if you could add me to the Responsible AI and the Security starter packs, similar to @mlamparth.bsky.social, I’m moving here from X 😊 120
Anka Reuel ➡️ NeurIPS @ankareuel.bsky.social · 06/12/2024In the same boat as @mlamparth.bsky.social, would appreciate if you could add me, too! Thanks so much 😊 120
Anka Reuel ➡️ NeurIPS @ankareuel.bsky.social · 06/12/2024In the same boat as @mlamparth.bsky.social, would appreciate if you could add me, too! Thanks so much 😊 110
Anka Reuel ➡️ NeurIPS @ankareuel.bsky.social · 06/12/2024In the same boat as @mlamparth.bsky.social, would appreciate if you could add me, too! Thanks so much 😊 110
Anka Reuel ➡️ NeurIPS @ankareuel.bsky.social · 05/12/2024You know it’s been a busy day when you realize, when taking out the trash, that that was literally the first time today you’ve stepped outside your house and away from your laptop🥲 (I love my work but time for a little unfiltered sunshine would’ve been great ☀️) 050
Anka Reuel ➡️ NeurIPS @ankareuel.bsky.social · 05/12/2024Thanks so much, this is great @ramshackle78.bsky.social! Also appreciate that you thought of my post a few days ago :) 110
Reposted by Anka Reuel ➡️ NeurIPSAnka Reuel ➡️ NeurIPS @ankareuel.bsky.social · 01/12/2024I’ll be at @neuripsconf.bsky.social in Vancouver from Dec 9 to Dec 15. Hit me up if you want to talk (non-)technical AI governance, science of evals, BetterBench, or just grab a coffee ☕️ #neurips2024 2152
Anka Reuel ➡️ NeurIPS @ankareuel.bsky.social · 03/12/2024Hang in there ❤️ Hope you can get some rest over the Christmas holidays if you celebrate! 110
Anka Reuel ➡️ NeurIPS @ankareuel.bsky.social · 01/12/2024For reference, I was one of the lead authors on the paper “Open Problems in Technical AI Governance”. While not a NeurIPS paper, I’m happy to talk about that, too, given that that field is where the majority of my current work falls under 😊 arxiv.org/abs/2407.14981arxiv.orgOpen Problems in Technical AI GovernanceAI progress is creating a growing range of risks and opportunities, but it is often unclear how they should be navigated. In many cases, the barriers and uncertainties faced are at least partly techni... 001
Anka Reuel ➡️ NeurIPS @ankareuel.bsky.social · 01/12/2024That’s exactly what I was looking for; thanks so much Keith! 000
Anka Reuel ➡️ NeurIPS @ankareuel.bsky.social · 01/12/2024I appreciate the counter-perspective! I’m trying to exactly understand that hype and whether it’s in any way justified/has materialized in the past few months! 000
Anka Reuel ➡️ NeurIPS @ankareuel.bsky.social · 01/12/2024I’ll be at @neuripsconf.bsky.social in Vancouver from Dec 9 to Dec 15. Hit me up if you want to talk (non-)technical AI governance, science of evals, BetterBench, or just grab a coffee ☕️ #neurips2024 2152
Anka Reuel ➡️ NeurIPS @ankareuel.bsky.social · 30/11/2024Thank you for the pointer, Lee! Much appreciated! 010
Anka Reuel ➡️ NeurIPS @ankareuel.bsky.social · 30/11/2024I’ll have a look, thanks for the pointer, AlinaG! 000
Anka Reuel ➡️ NeurIPS @ankareuel.bsky.social · 30/11/2024Thanks so much, @felixsimon.bsky.social, this is really helpful! And thanks @sethlazar.org for pinging Felix :) 120
Anka Reuel ➡️ NeurIPS @ankareuel.bsky.social · 30/11/2024Thanks for connecting me with the right people, @riana.bsky.social! 010
Anka Reuel ➡️ NeurIPS @ankareuel.bsky.social · 30/11/2024That’s already a helpful take, thanks @katestarbird.bsky.social! 010
Reposted by Anka Reuel ➡️ NeurIPSSeth Lazar @sethlazar.org · 25/11/2024Stellar work by some of the most promising young scholars I know of. Must read (and watch at Neurips). 151
Anka Reuel ➡️ NeurIPS @ankareuel.bsky.social · 30/11/2024That’s so kind, thank you for highlighting our work, Seth! 010