iiinova — where your digital self keeps evolving

Reposted by @aaditya6284.bsky.social

Stephanie Chan @scychan.bsky.social · 11/03/2025

New work led by @aaditya6284.bsky.social "Strategy coopetition explains the emergence and transience of in-context learning in transformers." We find some surprising things!! E.g. that circuits can simultaneously compete AND cooperate ("coopetition") 😯 🧵👇

294

aaditya6284.bsky.social @aaditya6284.bsky.social · 11/03/2025

Transformers employ different strategies through training to minimize loss, but how do these tradeoff and why? Excited to share our newest work, where we show remarkably rich competitive and cooperative interactions (termed "coopetition") as a transformer learns. Read on 🔎⏬

184

Reposted by @aaditya6284.bsky.social

Andrew Lampinen @lampinen.bsky.social · 10/12/2024

What counts as in-context learning (ICL)? Typically, you might think of it as learning a task from a few examples. However, we’ve just written a perspective (arxiv.org/abs/2412.03782) suggesting interpreting a much broader spectrum of behaviors as ICL! Quick summary thread: 1/7

arxiv.org

The broader spectrum of in-context learning

The ability of language models to learn a task from a few examples in context has generated substantial interest. Here, we provide a perspective that situates this type of supervised few-shot learning...

212232