iiinova — where your digital self keeps evolving

Tyler Chang

@tylerachang.bsky.social

230 followers 66 following 13 posts

PhD student at UC San Diego. He/him/his. tylerachang.github.io

Posts Replies Media

Reposted by Tyler Chang

Multilingual Representation Workshop @ EMNLP 2026 @mrl-workshop.bsky.social · 02/06/2026

We are releasing an expanded version of Global PIQA! It now covers 141 language varieties and includes parallel and non-parallel splits. We are also releasing an updated preprint.

Tyler Chang @tylerachang.bsky.social · 29/10/2025

Very very excited that Global PIQA is out! This was an incredible effort by 300+ researchers from 65 countries. The resulting dataset is a high-quality, participatory, and culturally-specific benchmark for over 100 languages.

Reposted by Tyler Chang

Catherine Arnett @catherinearnett.bsky.social · 19/09/2025

Did you know? ❌77% of language models on @hf.co are not tagged for any language 📈For 95% of languages, most models are multilingual 🚨88% of models with tags are trained on English In a new blog post, @tylerachang.bsky.social and I dig into these trends and why they matter! 👇

Reposted by Tyler Chang

Multilingual Representation Workshop @ EMNLP 2026 @mrl-workshop.bsky.social · 18/08/2025

We have over 200 volunteers now for 90+ languages! We are hoping to expand the diversity of our language coverage and are still looking for participants who speak these languages. Check out how to get involved below, and please help us spread the word!

We are still actively looking for volunteers speaking the following languages (or other languages not listed):
Afrikaans, Aymara, Basque, Bosnian, Breton, Burmese, Cebuano, Guarani, Haitian Creole, Hmong, Hungarian, Icelandic, Inuktitut, Irish, Karakalpak, Khmer, Kirghiz, Lao, Latvian, Macedonian, Malagasy, Maltese, Maori, Mongolian, Nahuatl, Navajo/Diné, Norwegian Nynorsk, Quechua, Romanian, Samoan, Scottish Gaelic, Shona, Somali, Tatar, Tibetan, Tigrinya, Waray, Walloon, Welsh, Yiddish, Zulu.

Reposted by Tyler Chang

Multilingual Representation Workshop @ EMNLP 2026 @mrl-workshop.bsky.social · 05/08/2025

With six weeks left before the deadline, we have had over 50 volunteers sign up to contribute for over 30 languages. If you don’t see your language represented on the map, this is your sign to get involved!

Tyler Chang @tylerachang.bsky.social · 25/06/2025

We're organizing a shared task to develop a multilingual physical commonsense reasoning evaluation dataset! Details on how to submit are at: sigtyp.github.io/st2025-mrl.h...

sigtyp.github.io

MRL 2025 Shared Task on Multilingual Physical Reasoning Datasets

Tyler Chang @tylerachang.bsky.social · 25/04/2025

of course, there are some scenarios where you would want to really check all the training examples, e.g. for detecting data contamination, or for rare facts, etc.

Tyler Chang @tylerachang.bsky.social · 25/04/2025

I think you could still make interesting inferences about what *types* of training examples influence the target! You'd essentially be getting a sample of the actual top-k retrievals

Tyler Chang @tylerachang.bsky.social · 25/04/2025

The biggest compute cost is computing gradients for every training example (~= cost of training) -- happy to chat more, especially if you know anyone interested in putting together an open-source implementation!

Tyler Chang @tylerachang.bsky.social · 25/04/2025

Presenting our work on training data attribution for pretraining this morning: iclr.cc/virtual/2025... -- come stop by in Hall 2/3 #526 if you're here at ICLR!

ICLR Poster Scalable Influence and Fact Tracing for Large Language Model PretrainingICLR 2025

Tyler Chang @tylerachang.bsky.social · 13/12/2024

And we hope you enjoy our paper: arxiv.org/abs/2410.17413 This work wouldn't have been at all possible without Dheeraj Rajagopal, Tolga Bolukbasi, @iislucas.bsky.social, and @iftenney.bsky.social !

Scalable Influence and Fact Tracing for Large Language Model Pretraining

Training data attribution (TDA) methods aim to attribute model outputs back to specific training examples, and the application of these methods to large language model (LLM) outputs could significantl...

Tyler Chang @tylerachang.bsky.social · 13/12/2024

Play with it yourself: see influential pretraining examples from our method for facts, factual errors, commonsense reasoning, arithmetic, and open-ended generation: github.com/PAIR-code/pr...

Tyler Chang @tylerachang.bsky.social · 13/12/2024

As models increase in size and pretraining tokens, "influence" more closely resembles "attribution". I.e. "better" models do seem to rely more on entailing examples.

Tyler Chang @tylerachang.bsky.social · 13/12/2024

Many influential examples do not entail a fact, but instead appear to reflect priors on common entities for certain relation types, or guesses based on first or last names.

Tyler Chang @tylerachang.bsky.social · 13/12/2024

In a fact tracing task, we find that classical retrieval methods (e.g. BM25) are still much better for retrieving examples that *entail* factual predictions (factual "attribution"), but TDA methods retrieve examples that have greater *influence* on model predictions.

Tyler Chang @tylerachang.bsky.social · 13/12/2024

Our method, TrackStar, refines existing gradient-based approaches to scale to much larger settings: over 100x more queries and a 30x larger retrieval corpus than previous work at this model size.

Tyler Chang @tylerachang.bsky.social · 13/12/2024

We scaled training data attribution (TDA) methods ~1000x to find influential pretraining examples for thousands of queries in an 8B-parameter LLM over the entire 160B-token C4 corpus! medium.com/people-ai-re...

Reposted by Tyler Chang

Catherine Arnett @catherinearnett.bsky.social · 22/11/2024

The Goldfish models were trained on byte-premium-scaled dataset sizes, such that if a language needs more bytes to encode a given amount of information, we scaled up the dataset according the byte premium. Read about how we (@tylerachang.bsky.social) trained the models: arxiv.org/pdf/2408.10441

Reposted by Tyler Chang

Catherine Arnett @catherinearnett.bsky.social · 15/11/2024

Tyler Chang and my paper got awarded outstanding paper at #EMNLP2024! Thanks to the award committee for the recognition!