iiinova — where your digital self keeps evolving

Reposted by Anka Reuel ➡️ NeurIPS

Cas (Stephen Casper) @scasper.bsky.social · 21/04/2025

🚨New paper: Current reports on AI audits/evals often omit crucial details, and there are huge disparities between the thoroughness of different reports. Even technically rigorous evals can offer little useful insight if reported selectively or obscurely. Audit cards can help.

142

Reposted by Anka Reuel ➡️ NeurIPS

Stanford HAI @stanfordhai.bsky.social · 25/03/2025

A recent Stanford paper reveals that many popular AI benchmarks are fundamentally flawed: They can be outdated, easily gamed, or inaccurate. Stanford HAI Graduate Fellow @ankareuel.bsky.social talks about how researchers are rethinking AI benchmarks: www.emergingtechbrew.com/stories/2025...

emergingtechbrew.com

Some researchers are rethinking how to measure AI intelligence

1103

Anka Reuel ➡️ NeurIPS @ankareuel.bsky.social · 27/01/2025

Submitting a benchmark to ICML? Check out our NeurIPS Spotlight paper BetterBench! We outline best practices for benchmark design, implementation & reporting to help shift community norms. Be part of the change! 🙌 + Add your benchmark to our database for visibility: betterbench.stanford.edu

1133

Anka Reuel ➡️ NeurIPS @ankareuel.bsky.social · 05/01/2025

📢 Excited to share: I'm again leading the efforts for the Responsible AI chapter for Stanford's 2025 AI Index, curated by @stanfordhai.bsky.social. As last year, we're asking you to submit your favorite papers on the topic for consideration (including your own!) 🧵 1/

1148

Anka Reuel ➡️ NeurIPS @ankareuel.bsky.social · 04/01/2025

I‘m teaching my first own course starting next week (Intro to AI Governance at Stanford). Super proud but also nervous 🥹 Any advice from more seasoned instructors? 😬 #AcademicTwitter #AcademicChatter #TeachingTips #AcademicAdvice

2110

Reposted by Anka Reuel ➡️ NeurIPS

Stephan Geering @stephangeering.bsky.social · 24/12/2024

The regular reminder of my starter packs full of amazing folks / accounts to follow. I am trying to keep them up to date but let me know if I missed you.

051

Anka Reuel ➡️ NeurIPS @ankareuel.bsky.social · 19/12/2024

As one of the vice chairs of the EU GPAI Code of Practice process, I co-wrote the second draft which just went online – feedback is open until mid-January, please let me know your thoughts, especially on the internal governance section! digital-strategy.ec.europa.eu/en/library/s...

digital-strategy.ec.europa.eu

Second Draft of the General-Purpose AI Code of Practice published, written by independent experts

Independent experts present the second draft of the General-Purpose AI Code of Practice, based on the feedback received on the first draft, published on 14 November 2024.

0145

Reposted by Anka Reuel ➡️ NeurIPS

Stanford HAI @stanfordhai.bsky.social · 11/12/2024

In our latest brief, Stanford scholars present a novel assessment framework for evaluating the quality of AI benchmarks and share best practices for minimum quality assurance. @ankareuel.bsky.social @chansmi.bsky.social @mlamparth.bsky.social hai.stanford.edu/what-makes-g...

0124

Anka Reuel ➡️ NeurIPS @ankareuel.bsky.social · 05/12/2024

You know it’s been a busy day when you realize, when taking out the trash, that that was literally the first time today you’ve stepped outside your house and away from your laptop🥲 (I love my work but time for a little unfiltered sunshine would’ve been great ☀️)

050

Reposted by Anka Reuel ➡️ NeurIPS

Anka Reuel ➡️ NeurIPS @ankareuel.bsky.social · 01/12/2024

I’ll be at @neuripsconf.bsky.social in Vancouver from Dec 9 to Dec 15. Hit me up if you want to talk (non-)technical AI governance, science of evals, BetterBench, or just grab a coffee ☕️ #neurips2024

2152

Anka Reuel ➡️ NeurIPS @ankareuel.bsky.social · 01/12/2024

2152

Reposted by Anka Reuel ➡️ NeurIPS

Seth Lazar @sethlazar.org · 25/11/2024

Stellar work by some of the most promising young scholars I know of. Must read (and watch at Neurips).

151

Anka Reuel ➡️ NeurIPS @ankareuel.bsky.social · 30/11/2024

Does anyone know of any empirical studies of the actual impact of AI or AI-generated (mis-)information on elections? (Doesn’t have to be US-focused, any election would be great). Thanks!

8206

Reposted by Anka Reuel ➡️ NeurIPS

Dan Roy @roydanroy.bsky.social · 27/11/2024

NeurIPS Test of Time Awards: Generative Adversarial Nets Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, Yoshua Bengio Sequence to Sequence Learning with Neural Networks Ilya Sutskever, Oriol Vinyals, Quoc V. Le

631129

Reposted by Anka Reuel ➡️ NeurIPS

Max Lamparth, Ph.D. @mlamparth.bsky.social · 26/11/2024

New MIT Tech Review article about our recent NeurIPS spotlight paper on how to measure and improve benchmark quality aspects! Great collaboration with @ankareuel.bsky.social, Amelia Hardy, @chansmi.bsky.social, Malcolm Hardy, and Mykel Kochenderfer. www.technologyreview.com/2024/11/26/1...

technologyreview.com

The way we measure progress in AI is terrible

Many of the most popular benchmarks for AI models are outdated or poorly designed.

192

Anka Reuel ➡️ NeurIPS @ankareuel.bsky.social · 26/11/2024

Thrilled our NeurIPS Spotlight paper BetterBench is featured by MIT @technologyreview.com! 🎉 Article: bit.ly/3Zo1rgw Paper: bit.ly/4eMSZfw Website & Scores: betterbench.stanford.edu Please share widely & join us in setting new standards for better AI benchmarking! ❤️

1163

Anka Reuel ➡️ NeurIPS @ankareuel.bsky.social · 26/11/2024

Okay, I’m convinced, 🦋 is better. Just need to find all my favorite AI governance people here now. Can you tag yourself or accounts worth following on everything AI governance? Thanks ❤️

370

Anka Reuel ➡️ NeurIPS @ankareuel.bsky.social · 25/11/2024

🚨 NeurIPS 2024 Spotlight Did you know we lack standards for AI benchmarks, despite their role in tracking progress, comparing models, and shaping policy? 🤯 Enter BetterBench–our framework with 46 criteria to assess benchmark quality: betterbench.stanford.edu 1/x

413925