iiinova — where your digital self keeps evolving

💬Panel discussion with Sally Haslanger and Marjolein Lanzing: A philosophical perspective on algorithmic discrimination Is discrimination the right way to frame the issues of lang tech? Or should we answer deeper rooted questions? And how does tech fit in systems of oppression?

📄Undesirable Biases in NLP: Addressing Challenges of Measurement We also presented our own work on strategies for testing the validity and reliability of LM bias measures: www.jair.org/index.php/ja...

Screenshot of a slide discussing how to improve how we communicate bias scores on Model Cards.

🔑Keynote @zeerak.bsky.social: On the promise of equitable machine learning technologies Can we create equitable ML technologies? Can statistical models faithfully express human language? Or are tokenizers "tokenizing" people—creating a Frankenstein monster of lived experiences?

Photo of the presentation. The slide shows an image of Frankenstein's monster.

162

📄A Capabilities Approach to Studying Bias and Harm in Language Technologies @hellinanigatu.bsky.social introduced us to the Capabilities Approach and how it can help us better understand the social impact of language technologies—with case studies of failing tech in the Majority World.

📄Angry Men, Sad Women: Large Language Models Reflect Gendered Stereotypes in Emotion Attribution Flor Plaza discussed the importance of studying gendered emotional stereotypes in LLMs, and how collaborating with philosophers benefits work on bias evaluation greatly.

🔑Keynote by John Lalor: Should Fairness be a Metric or a Model? While fairness is often viewed as a metric, using integrated models instead can help with explaining upstream bias, predicting downstream fairness, and capturing intersectional bias.

100

📄A Decade of Gender Bias in Machine Translation Eva Vanmassenhove: how has research on gender bias in MT developed over the years? Important issues, like non-binary gender bias, now fortunately get more attention. Yet, fundamental problems (that initially seemed trivial) remain unsolved.

120

📄MBBQ: A Dataset for Cross-Lingual Comparison of Stereotypes in Generative LLMs Vera Neplenbroek presented a multilingual extension of the BBQ bias benchmark to study bias across English, Dutch, Spanish, and Turkish. "Multilingual LLMs are not necessarily multicultural!"

100

🔑Keynote by Dong Nguyen: When LLMs meet language variation: Taking stock and looking forward Non-standard language is often seen as noisy/incorrect data, but this ignores the reality of language. Variation should play a larger role in LLM developments and sociolinguistics can help!

100

Oskar van der Wal @ovdw.bsky.social · 08/09/2024

Last week, we organized the workshop "New Perspectives on Bias and Discrimination in Language Technology" 🤖 @uvahumanities.bsky.social @amsterdamnlp.bsky.social We're looking back at two inspiring days of talks, posters, and discussions—thanks to everyone who participated! wai-amsterdam.github.io

Photo of the poster session at the workshop.

1131

This is a friendly reminder that there are 7 days left for submitting your extended abstract to this workshop! (Since the workshop is non-archival, previously published work is welcome too. So consider submitting previous/future work to join the discussion in Amsterdam!)

001

Workshop: New Perspectives on Bias and Discrimination in Language Technology.

This workshop is organized by University of Amsterdam researchers Katrin Schulz, Leendert van Maanen, @wzuidema.bsky.social, Dominik Bachmann, and myself. More information on the workshop can be found on the website, which will be updated regularly. wai-amsterdam.github.io

wai-amsterdam.github.io

🌟The goal of this workshop is to bring together researchers from different fields to discuss the state of the art on bias measurement and mitigation in language technology and to explore new avenues of approach.

One of the central issues discussed in the context of the societal impact of language technology is that ML systems can contribute to discrimination. Despite efforts to address these issues, we are far from solving them.

We're super excited to host Dong Nguyen, John Lalor, @zeerak.bsky.social and @azjacobs.bsky.social as invited speakers at this workshop! Submit an extended abstract to join the discussions; either in a 20min talk or a poster session. 📝Deadline Call for Abstracts: 15 Sep, 2024

121

Workshop: New Perspectives on Bias and Discrimination in Language Technology.

Working on #bias & #discrimination in #NLP? Passionate about integrating insights from different disciplines? And do you want to discuss current limitations of #LLM bias mitigation work? 🤖 👋Join the workshop New Perspectives on Bias and Discrimination in Language Technology 4&5 Nov in #Amsterdam!

wai-amsterdam.github.io

154

Reposted by Oskar van der Wal

Luca Soldaini 🎀 @soldaini.net · 01/02/2024

release day release day 🥳 OLMo 1b +7b out today and 65b soon... OLMo accelerates the study of LMs. We release *everything*, from toolkit for creating data (Dolma) to train/inf code blog blog.allenai.org/olmo-open-la... olmo paper allenai.org/olmo/olmo-pa... dolma paper allenai.org/olmo/dolma-p...

blog.allenai.org

OLMo: Open Language Model

A State-Of-The-Art, Truly Open LLM and Framework

12814

Oskar van der Wal @ovdw.bsky.social · 01/02/2024

But exciting to see more work dedicated to sharing models, checkpoints, and training data to the (research) community!

Oskar van der Wal @ovdw.bsky.social · 28/01/2024

I look forward to debates in the philosophy of (techno)science by people more knowledgeable. I'd say we have some philosophical basis that people are capable of such tasks. But there is also sufficient reason to believe LLM≠human, so any trust in one does not automatically transfer to the other.

100

Oskar van der Wal @ovdw.bsky.social · 28/01/2024

That is not to say I am categorically against using LLMs as epistemic tools, but from my own experience as a bias and interpretability researcher I think we should be careful of potential biases/failure modes. If we are transparent about their use and potential issues, I could see LLMs being useful.

Oskar van der Wal @ovdw.bsky.social · 28/01/2024

I think it all boils down to the reliability+validity of the approach. We don't have good methodologies (yet) to assess these qualities for LLMs compared to simpler more interpretable techniques. And intuitively I think we have more reasons to trust (expert) human annotators—see also psychometrics.

200

A 🧵thread about strategies for improving social bias evaluations of LMs. #blueskAI 🤖 bsky.app/profile/ovdw...

041

Special thanks go to Dominik Bachmann (shared first-author) whose insights from the perspective of psychometrics not only helped shape this paper, but also my views of current AI fairness practices more broadly.

000

This paper has been long in the works and is the result of many discussions trying to bridge the worlds of NLP and psychometrics. I am grateful for my co-authors Dominik Bachmann, Alina Leidinger, Leendert van Maanen, @wzuidema.bsky.social, and Katrin Schulz.

111

If you use/develop bias measures, we encourage you to apply a psychometric lens for reliably measuring the construct of interest. We end our paper with guidelines for developing bias measurement tools, which complements excellent advice by Dev et al. @zeerak.bsky.social, Blodgett et al. and others!

100

Maybe you want to decouple gender bias from grammatical gender (see e.g., Limisiewicz & Mareček). Or—when comparing the bias of models of different sizes—to make sure model capability is not a confounding factor. Maybe smaller models do not respond well to prompting and appear to be less biased.

100

However, we believe that its flip side—divergent validity—deserves attention as well! Instead, we ask whether the bias measure is not too similar to another (easily confounded) measure or construct. We do not want to accidentally also measure something else!

Screenshot of a figure with the caption: "This figure illustrates the difference between convergent and divergent validity (see Section 4.2). In this example, the convergent validity is assessed by testing how related a gender bias measure is to another gender bias measure. The divergent validity, instead, is assessed by testing whether the gender bias measure is not strongly correlated with a measure for another, but easily confounded construct (e.g., grammatical gender)."

100

An obvious approach is to test the convergent validity: How well do our bias scores relate to other bias measures? And—more importantly—to the downstream harms of LMs? (Sometimes called predictive validity) See, for example, the work by Delobelle et al., Goldfarb-Tarrant et al., and others.

100

Ideally, one would test the validity by comparing one’s bias results with a gold standard (criterion validity). Unfortunately, we do not have access to this for something like model bias... But there exist alternative (weaker) strategies for validating bias benchmarks!

100

Construct validity: How sure are we that we measure what we actually want to measure (the construct)? Critical work by e.g., Gonen & Goldberg, Blodgett et al., Orgad & Belinkov shows many flaws that could hurt the validity. How do we design bias measures that actually measure what we want?

Screenshot of a table with the caption: "An overview of the types of construct validity we discuss in Section 4. Examples are given in the last column."

100

Relatedly, this interesting post comes with a compelling argument: not the size of a benchmark, but its reliability matters! We waste valuable resources when computing results for test items that do not contribute to the overall reliability of the dataset. bsky.app/profile/lcho...

111

Parallel-form reliability tests if different—but designed to be equivalent—versions of a measure are consistent. E.g., how consistent are different prompt formulations for evaluating the LM responses on the same bias dataset? Are LMs sensitive to minor changes to how the questions are phrased?

100

Reliability: How much precision can we get when applying the bias measure? How resilient is it to random measurement error? Naturally, we prefer measurement tools with a higher reliability! We discuss four forms of reliability we think can be applied easily to the NLP context.

Screenshot of a table with the caption: "Examples of the reliability types we discuss in Section 3. We specify, for each reliability type, across which variations (e.g., random seeds) the consistency is measured. In the last column, we provide examples of where these reliability types could be applied."

100

We discuss two important concepts that say something about the quality of bias measures: reliability and construct validity. For both, we discuss strategies for how to assess these in the NLP setting.

100

It's important to understand the difference! For instance, a twice-as-high bias score (operationalization) does not necessarily mean that the model is twice as biased (construct). Making this distinction allows us to be more explicit about our assumptions and conceptualizations.

100

Borrowing from psychometrics (a field specialized in the measurement of concepts that are not directly observable), we argue that it is useful to decouple the "construct" (what we want to know about but cannot observe directly) from its "operationalization" (the imperfect proxy).

Screenshot of a figure with the caption: "We assume that a training dataset's bias influences the bias of a model trained on that data (but other possible sources of bias are possible, e.g., model compression may amplify existing biases (Hooker et al., 2020)). Training dataset bias and model bias are unobservable constructs (circle) that both have different possible operationalizations (squares)."

111

Developing tools for measuring & mitigation is hard: LM bias is a complex sociocultural phenomenon + we have no access to a ground truth. We voice our concerns about current bias eval practices, and discuss how we can test the quality of bias measures despite these challenges.

100

Undesirable Biases in NLP: Addressing Challenges of Measurement | Journal of Artificial In...

I am super excited to share that our paper "Undesirable Biases in NLP: Addressing Challenges of Measurement" has been published in JAIR! doi.org/10.1613/jair...

doi.org

131

Oskar van der Wal @ovdw.bsky.social · 07/01/2024

Thanks for creating this, I'd like to be on the list!

Naomi Saphra @nsaphra.bsky.social · 05/01/2024

Reposted by Oskar van der Wal

I decided what we need to make blueskAI happen is a feed. Reply here to get added to the whitelist! Whitelisted users can post to the feed by adding the following keywords to a post: 🤖 bskAI blueskAI

624317

Oskar van der Wal @ovdw.bsky.social · 11/12/2023

If these kinds of questions excite you too, check out the paper! Code can be found here: github.com/iabhijith/bi... This work was done with Abhijith Chintam, Rahel Beloch, Willem Zuidema, and Michael Hanna

github.com

GitHub - iabhijith/bias-causal-analysis

Contribute to iabhijith/bias-causal-analysis development by creating an account on GitHub.