dchiang.bsky.social @dchiang.bsky.social · 30/10/2025I am recruiting a PhD student to work with me, Peter Cholak, Anand Pillay, and Andy Yang @pentagonalize.bsky.social on transformers and logic/model theory (or related topics). If you are interested, please email me with "FLaNN" in the subject line! 000
Reposted by @dchiang.bsky.socialpentagonalize.bsky.social @pentagonalize.bsky.social · 03/10/2025Read the cookbook: arxiv.org/abs/2510.00368 Join us for weekly seminars on formal language theory, ML, NLP, and more: flannseminars.github.io 012
Reposted by @dchiang.bsky.socialpentagonalize.bsky.social @pentagonalize.bsky.social · 03/10/2025Thanks to all the chefs: @ccwatson.bsky.social, @antonxue.bsky.social, @satwik77.bsky.social, @ll4r3n4.bsky.social, @lambdaviking.bsky.social, Emile Dos Santos Ferreira, @anejsvete.bsky.social, @dchiang.bsky.social 122
Reposted by @dchiang.bsky.socialpentagonalize.bsky.social @pentagonalize.bsky.social · 03/10/2025There is no better way to understand what transformers can do than to get your hands dirty and construct them, weight-by-weight. The Transformer Cookbook provides a guide for anyone aiming to understand the expressive power of transformers on such a formal level. 112
Reposted by @dchiang.bsky.socialpentagonalize.bsky.social @pentagonalize.bsky.social · 03/10/2025We present The Transformer Cookbook: a collection of recipes for programming algorithms directly into transformers! Hungry for an induction head? Craving a Dyck language recognizer? We show you step-by-step how to cook up transformers for these algorithms and many more!arxiv.orgThe Transformer CookbookWe present the transformer cookbook: a collection of techniques for directly encoding algorithms into a transformer's parameters. This work addresses the steep learning curve of such endeavors, a prob... 155
Reposted by @dchiang.bsky.socialarXiv cs.LG Machine Learning @cslg-bot.bsky.social · 02/10/2025Andy Yang, Christopher Watson, Anton Xue, Satwik Bhattamishra, Jose Llarena, William Merrill, Emile Dos Santos Ferreira, Anej Svete, David Chiang: The Transformer Cookbook arxiv.org/abs/2510.00368 arxiv.org/pdf/2510.00368 arxiv.org/html/2510.00368 002
dchiang.bsky.social @dchiang.bsky.social · 23/06/2025New on arXiv: Knee-Deep in C-RASP, by @pentagonalize.bsky.social, @cadilhac.bsky.social, and me. The solid stepped line is our theoretical prediction based on what problems C-RASP can solve, and the numbers/colors are what transformers (no position embedding) can learn. 121
dchiang.bsky.social @dchiang.bsky.social · 23/04/2025Congratulations to Aarohi Srivastava @aarsri.bsky.social on winning the Best Paper Award at W-NUT at NAACL 2025! This paper applies various interventions simulating noisy text or dialectal variation to discover how different interventions have different effects. 131
dchiang.bsky.social @dchiang.bsky.social · 18/03/2025The abstract submission deadline for Midwest Speech and Language Days is in two days, on March 20! Please submit an abstract! MSLD is non-archival, and submissions of both work-in-progress and previously published work are encouraged. nlp.nd.edu/msld25/nlp.nd.eduMidwest Speech and Language Days 2025 200
dchiang.bsky.social @dchiang.bsky.social · 08/03/2025Midwest Speech and Language Days will be held Apr 15-16 at @NotreDame! Abstract submissions are due Mar 20, and registration deadline is Mar 27. Financial assistance for students (lodging, poster printing) is available. nlp.nd.edu/msld25nlp.nd.eduMidwest Speech and Language Days 2025 102
dchiang.bsky.social @dchiang.bsky.social · 23/12/2024New paper and two not-so-new papers on arXiv about transformer expressivity: (1) With @pentagonalize and Dana Angluin, "Simulating Hard Attention Using Soft Attention" arxiv.org/abs/2412.09925arxiv.orgSimulating Hard Attention Using Soft AttentionWe study conditions under which transformers using soft attention can simulate hard attention, that is, effectively focus all attention on a subset of positions. First, we examine several variants of ... 231
