iiinova — where your digital self keeps evolving

Reposted by @prakharg.bsky.social

AFAA 2026 @ ICLR @afciworkshop.bsky.social · 02/02/2026

🚨 Deadline Extended to Feb 5 (AoE)! CFP still OPEN for the #AFAA2026 Workshop at @iclr-conf.bsky.social — on fairness across alignment & agentic AI systems. Full & tiny papers welcome • Interdisciplinary work encouraged! 🔗 afciworkshop.org #ICLR2026 #AFAA2026

afciworkshop.org

AFAA 2026

The Algorithmic Fairness Across Alignment Procedures and Agentic Systems (AFAA) workshop aims to spark discussions on rethinking fairness in AI alignment procedures and agentic system development.

021

Reposted by @prakharg.bsky.social

AFAA 2026 @ ICLR @afciworkshop.bsky.social · 06/01/2026

🚨 CFP OPEN! We’re launching the #AFAA2026 Workshop at @iclr-conf.bsky.social on 𝗳𝗮𝗶𝗿𝗻𝗲𝘀𝘀 𝗮𝗰𝗿𝗼𝘀𝘀 𝗮𝗹𝗶𝗴𝗻𝗺𝗲𝗻𝘁 𝗮𝗻𝗱 𝗮𝗴𝗲𝗻𝘁𝗶𝗰 𝗔𝗜 𝘀𝘆𝘀𝘁𝗲𝗺𝘀. Submit your latest ideas (full or tiny papers!) Interdisciplinary work especially welcome :D 🗓 Deadline: Jan 31 (AoE) | 🔗 www.afciworkshop.org #AFAA2026 #ICLR2026

afciworkshop.org

AFAA 2026

The Algorithmic Fairness Across Alignment Procedures and Agentic Systems (AFAA) workshop aims to spark discussions on rethinking fairness in AI alignment procedures and agentic system development.

062

prakharg.bsky.social @prakharg.bsky.social · 25/06/2025

Four case studies with the gap between the reality of model use and their sandbox evaluations in audits... Definitely need to take a deeper dive, great presentation by Emily Black!

000

prakharg.bsky.social @prakharg.bsky.social · 25/06/2025

Evaluations in the way the model would be deployed vs evaluations in only controlled unrealistic settings!

100

prakharg.bsky.social @prakharg.bsky.social · 25/06/2025

Allowing companies to do isolated audits can lead to D-Hacking!! More robust testing is needed...

100

prakharg.bsky.social @prakharg.bsky.social · 25/06/2025

Legal frameworks tend to have control over allocative decisions (Yes/No outcomes), which fit well with traditional ML systems... But not with GenAI systems

100

prakharg.bsky.social @prakharg.bsky.social · 25/06/2025

Zollo et al: Towards Effective Discrimination Testing for Generative AI #FAccT2025

110

prakharg.bsky.social @prakharg.bsky.social · 25/06/2025

Nuance of stereotype errors is so important to understand their true harms... Insightful presentation by @angelinawang.bsky.social

000

prakharg.bsky.social @prakharg.bsky.social · 25/06/2025

Women tend to report stereotype-reinforcing errors as more harmful while men tend to report stereotype-violating errors as more harmful...

100

prakharg.bsky.social @prakharg.bsky.social · 25/06/2025

Some items are more associated with men vs women (not surprising), but not all of them are equally harmful!!

100

prakharg.bsky.social @prakharg.bsky.social · 25/06/2025

Cognitive beliefs, attitudes and behaviours... Three ways to measure harms ('pragmatic harms')

100

prakharg.bsky.social @prakharg.bsky.social · 25/06/2025

Are all errors equally harmful? No! Stereotype-reinforcing errors vs stereotype-violating errors

100

prakharg.bsky.social @prakharg.bsky.social · 25/06/2025

Our understanding of stereotypes sometimes isn't indicative of reality.... they can appear in both directions, or might exist simply without harm

100

prakharg.bsky.social @prakharg.bsky.social · 25/06/2025

Wang et al: Measuring Machine Learning Harms from Stereotypes Requires Understanding Who Is Harmed by Which Errors in What Ways #FAccT2025

110

prakharg.bsky.social @prakharg.bsky.social · 25/06/2025

Clear narrative and a great presentation by Cecilia Panigutti

000

prakharg.bsky.social @prakharg.bsky.social · 25/06/2025

Risk-measuring studies - Bringing it back to risk measurement, but this time with a clearly defined objective instead of risk-uncovering as before... Not just whether a risk exists, but 'how severe' is it?

100

prakharg.bsky.social @prakharg.bsky.social · 25/06/2025

Interface-design studies - Focus on UI design elements which impact user interaction

100

prakharg.bsky.social @prakharg.bsky.social · 25/06/2025

Reverse-engineering studies - Narrower scope and in-depth studies of how algorithms work... Methodological precision in the key!

100

prakharg.bsky.social @prakharg.bsky.social · 25/06/2025

Risk-uncovering studies - Typical starts from anecdotal evidence and help surface new risks

100

prakharg.bsky.social @prakharg.bsky.social · 25/06/2025

A review organized not by data collection technique, but by DSA risk management framework categories

100

prakharg.bsky.social @prakharg.bsky.social · 25/06/2025

Narrative review of algorithmic auditing studies, practical recommendation for best practices, and mapping to DSA obligations...

100

prakharg.bsky.social @prakharg.bsky.social · 25/06/2025

Panigutti et al: How to investigate algorithmic-driven risks in online platforms and search engines? A narrative review through the lens of the EU Digital Services Act #FAccT2025

100

prakharg.bsky.social @prakharg.bsky.social · 25/06/2025

Historical methods working alongside many other ways of auditing these models can help us take advantage of the broader scope of historical evaluations....

100

prakharg.bsky.social @prakharg.bsky.social · 25/06/2025

AI Audits have moved from bottom-up external evaluations to new age 'auditing companies'. While this has increased speed and scale, they have significantly narrowed the scope of auditing.

100

prakharg.bsky.social @prakharg.bsky.social · 25/06/2025

Why the history of AI assessments? A study through the lens of historical methods can help us understand neglected areas of auditing.

100

prakharg.bsky.social @prakharg.bsky.social · 25/06/2025

Sandoval and Jing: Historical Methods for AI Evaluations, Assessments, and Audits #FAccT2025

100

prakharg.bsky.social @prakharg.bsky.social · 25/06/2025

Important recommendations on standardization of report creation and storage to allow better meta-analysis in the future... Eye opening presentation by @mkgerchick.bsky.social

000

prakharg.bsky.social @prakharg.bsky.social · 25/06/2025

Applicants impacted by these tools, whose demographic data is missing, are completely removed from these audits!

100

prakharg.bsky.social @prakharg.bsky.social · 25/06/2025

Serious issues with the data usage... most weird for me: 'simulated test data'!

100

prakharg.bsky.social @prakharg.bsky.social · 25/06/2025

More than 98% Fortune 500 companies use some form of automated hiring, only about 2% of them have audited these systems!!

100

prakharg.bsky.social @prakharg.bsky.social · 25/06/2025

NYC LL 144: One of the first enacted laws in the US regulating the use of AI in employment

100

prakharg.bsky.social @prakharg.bsky.social · 25/06/2025

Gerchick et al: Auditing the Audits: Lessons for Algorithmic Accountability from Local Law 144's Bias Audits #FAccT2025

121

prakharg.bsky.social @prakharg.bsky.social · 25/06/2025

Target Identification -> Prompt collection -> Prompt perturbations -> Response collection -> Response evaluation

110

prakharg.bsky.social @prakharg.bsky.social · 25/06/2025

Don't freely perturb your prompts using LLMs, unless you want random 'Yo's in your text :P Be more structured and intentional!

110

prakharg.bsky.social @prakharg.bsky.social · 25/06/2025

Most existing dialect bias auditing in LLMs tend to rely on existing corpora which doesn't really work for chatbots, hence the need for a new framework...

110

prakharg.bsky.social @prakharg.bsky.social · 25/06/2025

Dialect biases in LLMs... But specifically for chatbots, and a case study on Amazon Rufus

110

prakharg.bsky.social @prakharg.bsky.social · 25/06/2025

Harvey et al: A Framework for Auditing Chatbots for Dialect-Based Quality-of-Service Harms #FAccT2025

110

prakharg.bsky.social @prakharg.bsky.social · 25/06/2025

These communities are not 'hard-to-reach', they just require context-aware techniques.. An amazing presentation by Ankolika De!

000

prakharg.bsky.social @prakharg.bsky.social · 25/06/2025

Even the IRB approved methods and recommendations were severely in clash with local practices!! Such an important lesson on rigid Ethics infrastructures that fail in underrepresented contexts in the community

100

prakharg.bsky.social @prakharg.bsky.social · 25/06/2025

A case study on WhatsApp's roles in Business and Politics in India...

100

prakharg.bsky.social @prakharg.bsky.social · 25/06/2025

Context-aware approaches to auditing.... Dominant techniques don't work in less explored contexts like the Global South

100

prakharg.bsky.social @prakharg.bsky.social · 25/06/2025

De et al: Who Gets Heard? Calling Out the "Hard-to-Reach" Myth for Non-WEIRD Populations' Recruitment and Involvement in Research #FAccT2025

101

prakharg.bsky.social @prakharg.bsky.social · 25/06/2025

US and Germany are the most audited contexts (more than 70% of all audits!!).. Such an intriguing presentation by @aurman21.bsky.social

021

prakharg.bsky.social @prakharg.bsky.social · 25/06/2025

Algorithmic audit history started from price discrimination (never knew that!) to a focus on the distribution of harmful content online... Most popular targets seem to be search and recommendation

100

prakharg.bsky.social @prakharg.bsky.social · 25/06/2025

Algorithmic auditing needs to be geographically and linguistically situated! Or else, we are generalizing results in situations that doesn't make sense...

100

prakharg.bsky.social @prakharg.bsky.social · 25/06/2025

What are the questions in Algorithmic Audits that are NOT answered?

100

prakharg.bsky.social @prakharg.bsky.social · 25/06/2025

Urman et al: WEIRD Audits? Research Trends, Linguistic and Geographical Disparities in the Algorithm Audits of Online Platforms - A Systematic Literature Review #FAccT2025

100

prakharg.bsky.social @prakharg.bsky.social · 25/06/2025

'A crisis of representation in AI Ethics'... A very reflective presentation by Abdullah Hasan Safir.

010