iiinova — where your digital self keeps evolving

technion-cs-nlp.github.io

In this work we take a step towards understanding and mitigating the vision-language performance gap, but there's still more to explore! This was an awesome collaboration w\ Yossi Gandelsman, @boknilev.bsky.social, led by Yaniv Nikankin 🤩 Paper and code: technion-cs-nlp.github.io/vlm-circuits...

Same Task, Different Circuits – Project Page

010 0

By simply patching visual data tokens from later layers back into earlier ones, we improve of 4.6% on average - closing a third of the gap!

4. Zooming on data positions, we show that visual representations gradually align with their textual analogs across model layers (also shown by @zhaofeng_wu et al.). We hypothesize this may happen too late in the model to process the information, and fix it with back-patching.

3. Data sub-circuits, however, are modality-specific; Swapping them significantly degrades performance. This is critical - this highlights that the differences in data processing are a key factor in the performance gap.

2. Structure is only half the story: different circuits can still implement similar logic. We swap sub-circuits between modalities to measure cross-modal faithfulness. Turns out, query and generation sub-circuits are functionally equivalent, retaining faithfulness when swapped!

1. Circuits for the same task are mostly structurally disjoint, with an average of only 18% components shared between modalities! The overlap is extremely low in data and query positions, and moderate in the generation (last) position only.

We identify circuits (task-specific computational sub-graphs composed of attention heads and MLP neurons) used by VLMs to solve both variants. What did we find? >>

Consider object counting: we can ask a VLM “how many books are there?” given either an image or a sequence of words. Like Kaduri et al., we consider three types of positions within the input - data (image or word sequence), query ("how many..."), and generation (last token).

BlackboxNLP @blackboxnlp.bsky.social · 24/06/2025

VLMs perform better on questions about text than when answering the same questions about images - but why? and how can we fix it? In a new project led by Yaniv (@YNikankin on the other app), we investigate this gap from an mechanistic perspective, and use our findings to close a third of it! 🧵

164 0

Reposted by Dana Arad

Working on circuit discovery in LMs? Consider submitting your work to the MIB Shared Task, part of #BlackboxNLP at @emnlpmeeting.bsky.social 2025! The goal: benchmark existing MI methods and identify promising directions to precisely and concisely recover causal pathways in LMs >>

154 0

Reposted by Dana Arad

BlackboxNLP @blackboxnlp.bsky.social · 23/06/2025

Have you heard about this year's shared task? 📢 Mechanistic Interpretability (MI) is quickly advancing, but comparing methods remains a challenge. This year at #BlackboxNLP, we're introducing a shared task to rigorously evaluate MI methods in language models 🧵

1164 0

Reposted by Dana Arad

Aaron Mueller @amuuueller.bsky.social · 27/05/2025

SAEs have been found to massively underperform supervised methods for steering neural networks. In new work led by @danaarad.bsky.social, we find that this problem largely disappears if you select the right features!

1161 0

Dana Arad @danaarad.bsky.social · 28/05/2025

Thank you! Added to my reading list ☺️

010 0

SAEs Are Good for Steering -- If You Select the Right Features

SAEs have sparked a debate over their utility; we hope to add another perspective. Would love to hear your thoughts! Paper: arxiv.org/abs/2505.20063 Code: github.com/technion-cs-... Huge thanks to ‪@boknilev.bsky.social‬, ‪@amuuueller.bsky.social‬, it’s been great working on this project with you!

arxiv.org

Sparse Autoencoders (SAEs) have been proposed as an unsupervised approach to learn a decomposition of a model's latent space. This enables useful applications such as steering - influencing the output...

150 0

These findings have practical implications: after filtering out features with low output scores, we see 2-3x improvements for steering with SAEs, making them competitive with supervised methods on AxBench, a recent steering benchmark ( Wu and ‪@aryaman.io‬ et al.)

We show that high scores rarely co-occur, and emerge at different layers: features in earlier layers primarily detect input patterns, while features in later layers are more likely to drive the model’s outputs, consistent with prior analyses of LLM neuron functionality.

These differences were previously noted (e.g., Durmus et al., see image), but had not been systematically analyzed. We take an additional step by introducing two simple, efficient metrics to characterize features: the input score and the output score.

In this work we characterize two feature roles: Input features, which mainly capture patterns in the model's input, and output features, those with a human-understandable effect on the model's output. Steering with each yields very different effects!