iiinova — where your digital self keeps evolving

Isabel Papadimitriou @isabelpapad.bsky.social · 17/09/2025

It’s a very exciting time to be thinking about the interaction of vision and language, and what we can find in (and learn from) VLMs. Looking forward to talking to people about this at COLM, and thanks to everyone doing awesome research on this topic!

110

Isabel Papadimitriou @isabelpapad.bsky.social · 17/09/2025

Lastly, we didn’t just go blindly into batchtopk SAEs, we tried other SAEs and a semi-NMF, but they don’t work as well: batchtopk dominates the reconstruction-sparsity tradeoff

110

Isabel Papadimitriou @isabelpapad.bsky.social · 17/09/2025

Check out our interactive demo (by the amazing @napoolar), where bridges illustrate our BridgeScore metric: a combination geometrical alignment (cosine) and statistical alignment (coactivation on image-caption pairs): vlm-concept-visualization.com

120

Isabel Papadimitriou @isabelpapad.bsky.social · 17/09/2025

And they’re stable ~across training data mixtures~! If we train the SAEs with a 5:1 ratio of text to images, we get a lot more text concepts (makes sense!). But if we weight the points by activation scores (bottom), we see basically the same concepts across very different mixtures

110

Isabel Papadimitriou @isabelpapad.bsky.social · 17/09/2025

But, are the SAEs even stable? It wouldn’t be very enlightening if we were just analyzing a fluke of the SAE seed. Across seeds, we find that frequently-used concepts (the ones that take up 99% of activation weights) are remarkably stable, but the rest are pretty darn unstable.

100

Isabel Papadimitriou @isabelpapad.bsky.social · 17/09/2025

How can this be? Because of the projection effect in SAEs! When we impose sparisty, then the inputs that are activated don’t necessarily reflect the whole story of what inputs align with that direction. Here, the batchtopk cutoff (dotted line) hides a multimodal story

110

Isabel Papadimitriou @isabelpapad.bsky.social · 17/09/2025

On first blush, however, the concepts look pretty single-modality: see here their modality scores (how many of the top-activating inputs are images vs text). The classifier results above show us that the actual geometry is often much closer to modality-agnostic.

110

Isabel Papadimitriou @isabelpapad.bsky.social · 17/09/2025

In fact, they often can’t even act as good modality classifiers: if we take the SAE concept direction, and see how well projecting on to that direction separates modality, we see that many of the concepts don’t get great accuracy

110

Isabel Papadimitriou @isabelpapad.bsky.social · 17/09/2025

We trained SAEs on the embedding spaces of four VLMs, and analyzed the resulting dictionaries of concepts. Even though image and text concepts lie on separate anisotropic cones, the SAE concepts don’t lie within those cones.

100

Isabel Papadimitriou @isabelpapad.bsky.social · 17/09/2025

Are there conceptual directions in VLMs that transcend modality? Check out our COLM oral spotlight 🔦 paper! We use SAEs to analyze the multimodality of linear concepts in VLMs with @chloesu07.bsky.social, @thomasfel.bsky.social, @shamkakade.bsky.social and Stephanie Gil arxiv.org/abs/2504.11695

1256

Reposted by Isabel Papadimitriou

Ben Prystawski @benpry.bsky.social · 01/08/2025

How do people trade off between speed and accuracy in reasoning tasks without easy heuristics? Come to my talk, "Thinking fast, slow, and everywhere in between in humans and language models," in the Reasoning session this afternoon #CogSci2025 to find out! paper: escholarship.org/uc/item/5td9...

escholarship.org

Thinking fast, slow, and everywhere in between in humans and language models

Author(s): Prystawski, Ben; Goodman, Noah | Abstract: How do humans adapt how they reason to varying circumstances? Prior research has argued that reasoning comes in two types: a fast, intuitive type ...

051

Reposted by Isabel Papadimitriou

Ben Prystawski @benpry.bsky.social · 01/08/2025

When people form conventions in reference games, how easy are they for outsiders to interpret? (for values of "outsider" that include naïve humans and vision-language models) Check out @vboyce.bsky.social's poster today at #CogSci2025 to find out. paper: escholarship.org/uc/item/16c4...

escholarship.org

Idiosyncratic but not opaque: Linguistic conventions formed in reference games are interpretable by naÃ¯ve humans and visionâ€“language models

Author(s): Boyce, Veronica; Prystawski, Ben; Tan, Alvin Wei Ming; Frank, Michael C. | Abstract: When are in-group linguistic conventions opaque to non-group members (teen slang like "rizz") or general...

063

Isabel Papadimitriou @isabelpapad.bsky.social · 18/06/2025

@antararb.bsky.social is applying for PhDs this fall! She’s super impressive and awesome to work with, and conceived of this project independently and carried it out very successfully! Keep an eye out 🙂

010

Isabel Papadimitriou @isabelpapad.bsky.social · 18/06/2025

So is it really this implicit operators thing that’s tripping them up? We try many other ablations, looking at the effect of giving extra context in the prompt, using numbers vs words, left-to-right ordering, and subtractive systems, and none of them seem to affect the models that much.

110

Isabel Papadimitriou @isabelpapad.bsky.social · 18/06/2025

Our experiments are based on Linguistics Olympiad problems that deal with number systems, like the one here. We created additional hand-standardized versions of each puzzle in order to be able to do all of the operator ablations.

110

Isabel Papadimitriou @isabelpapad.bsky.social · 18/06/2025

This shows the types of reasoning and variable binding jumps that are hard for LMs. It’s hard to go one level up, and bind a variable to have the meaning of an operator, or to understand that an operator is implicit.

110

Isabel Papadimitriou @isabelpapad.bsky.social · 18/06/2025

If we alter the problems to make the operators explicit, the models can solve these problems pretty easily. But it’s still harder to bind a random symbol or word to mean an operator like +. It’s much easier when we use the familiar symbols for the operators, like + and x.

110

Isabel Papadimitriou @isabelpapad.bsky.social · 18/06/2025

Our main finding: LMs find it hard when *operators* are implicit. We don’t say “5 times 100 plus 20 plus 3”, we say “five hundred and twenty-three”. The Linguistics Olympiad puzzles are pretty simple systems of equations that an LM should solve – but the operators aren’t explicit.

110

Isabel Papadimitriou @isabelpapad.bsky.social · 18/06/2025

Why can’t LMs solve puzzles about the number systems of languages, when they can solve really complex math problems? Our new paper, led by @antararb.bsky.social looks at why this intersection of language and math is difficult, and what this means for LM reasoning! arxiv.org/abs/2506.13886

4284

Reposted by Isabel Papadimitriou

Naomi Saphra @nsaphra.bsky.social · 12/06/2025

ACL paper alert! What structure is lost when using linearizing interp methods like Shapley? We show the nonlinear interactions between features reflect structures described by the sciences of syntax, semantics, and phonology.

35412

Reposted by Isabel Papadimitriou

Mike Frank @mcxfrank.bsky.social · 30/05/2025

Congrats to Veronica Boyce on her dissertation defense! That’s three amazing talks by three great students in 8 days!

0301

Isabel Papadimitriou @isabelpapad.bsky.social · 06/03/2025

(the unfortunate truth is that I am really enjoying this mac and its battery life oops)

010

Isabel Papadimitriou @isabelpapad.bsky.social · 06/03/2025

This work Mac (my first ever) is great because every time something seriously breaks, instead of becoming distressed and despondent like I usually do, it's just like "ooooooh yeahhh, yet another win for team Linux 😎😎😎🎉🐧"

170

Reposted by Isabel Papadimitriou

iseeaswell.bsky.social @iseeaswell.bsky.social · 19/02/2025

😼SMOL DATA ALERT! 😼Anouncing SMOL, a professionally-translated dataset for 115 very low-resource languages! Paper: arxiv.org/pdf/2502.12301 Huggingface: huggingface.co/datasets/goo...

2148

Reposted by Isabel Papadimitriou

Tobias Gerstenberg @tobigerstenberg.bsky.social · 12/02/2025

New paper in Psychological Review! In "Causation, Meaning, and Communication" Ari Beller (cicl.stanford.edu/member/ari_b...) develops a computational model of how people use & understand expressions like "caused", "enabled", and "affected". 📃 osf.io/preprints/ps... 📎 github.com/cicl-stanfor... 🧵

15717

Isabel Papadimitriou @isabelpapad.bsky.social · 16/01/2025

Where are all of the phoneticians of the Boston area and why isn't there a storied subfield of fieldwork studying the Cambridge shopkeeper who seems to have a mix between a West Country (rhotic English!) and a Boston (non-rhotic American!) accent. Apparently the shop's been open for decades, smh

010

Reposted by Isabel Papadimitriou

Kyle Mahowald @kmahowald.bsky.social · 13/01/2025

Quanta write-up of our Mission: Impossible Language Models work, led by @juliekallini.bsky.social. As the photos suggest, Richard, @isabelpapad.bsky.social, and I do all our work sitting together around a single laptop and pointing at the screen.

0264

Isabel Papadimitriou @isabelpapad.bsky.social · 17/12/2024

My most controversial take is that you should never use commit -m, just let it open the damn vim file, let yourself think for a second, and then write something descriptive

1111

Isabel Papadimitriou @isabelpapad.bsky.social · 10/12/2024

I will be at NeurIPS starting tomorrow! Would love to chat about interpretability, linguistics, language structure, meaning in LLMs. Reach out! Aaaand if you love Vancouver, apply to do a PhD at UBC! Fun research in a lovely place! linguistics.ubc.ca/graduate/adm...

091

Isabel Papadimitriou @isabelpapad.bsky.social · 18/11/2024

Do you want to understand how language models work, and how they can change language science? I'm recruiting PhD students at UBC Linguistics! The research will be fun, and Vancouver is lovely. So much cool NLP happening at UBC across both Ling and CS! linguistics.ubc.ca/graduate/adm...

Aerial picture of the UBC campus, with an arrow pointing to a building and text asking "Your PhD lab?"

1227