iiinova — where your digital self keeps evolving

Thomas Fel @thomasfel.bsky.social · 15/10/2025

That concludes this two-part descent into the Rabbit Hull. Huge thanks to all collaborators who made this work possible — and especially to @binxuwang.bsky.social , with whom this project was built, experiment after experiment. 🎮 kempnerinstitute.github.io/dinovision/ 📄 arxiv.org/pdf/2510.08638

050

Thomas Fel @thomasfel.bsky.social · 15/10/2025

If this holds, three implications: (i) Concepts = points (or regions), not directions (ii) Probing is bounded: toward archetypes, not vectors (iii) Can't recover generating hulls from sum: we should look deeper than just a single-layer activations to recover the true latents

111

Thomas Fel @thomasfel.bsky.social · 15/10/2025

Synthesizing these observations, we propose a refined view, motivated by Gärdenfors' theory and attention geometry. Activations = multiple convex hulls simultaneously: a rabbit among animals, brown among colors, fluffy among textures. The Minkowski Representation Hypothesis.

120

Thomas Fel @thomasfel.bsky.social · 15/10/2025

Taken together, the signs of partial density, local connectedness, and coherent dictionary atoms indicate that DINO’s representations are organized beyond linear sparsity alone.

110

Thomas Fel @thomasfel.bsky.social · 15/10/2025

Can position explain this ? We found that pos. information collapses: from high-rank to a near 2-dim sheet. Early layers encode precise location; later ones retain abstract axes. This compression frees dimensions for features, and *position doesn't explain PCA map smoothness*

100

Thomas Fel @thomasfel.bsky.social · 15/10/2025

Patch embeddings form smooth, connected surfaces tracing objects and boundaries. This may suggests interpolative geometry: tokens as mixtures between landmarks, shaped by clustering and spreading forces in the training objectives.

110

Thomas Fel @thomasfel.bsky.social · 15/10/2025

We found antipodal feature pairs (dᵢ ≈ − dⱼ): vertical vs horizontal lines, white vs black shirts, left vs right… Also, co-activation statistics only moderately shape geometry: concepts that fire together aren't necessarily nearby—nor orthogonal when they don't.

100

Thomas Fel @thomasfel.bsky.social · 15/10/2025

Under the Linear Rep. Hypothesis, we'd expect Dictionary to be quasi-orthogonality. Instead, training drives atoms from near-Grassmannian initialization to higher coherence. Several concepts fire almost always the embedding is partly dense (!), contradicting pure sparse coding.

110

Thomas Fel @thomasfel.bsky.social · 15/10/2025

🕳️🐇Into the Rabbit Hull – Part II Continuing our interpretation of DINOv2, the second part of our study concerns the *geometry of concepts* and the synthesis of our findings toward a new representational *phenomenology*: the Minkowski Representation Hypothesis

2339

Thomas Fel @thomasfel.bsky.social · 14/10/2025

Huge thanks to all collaborators who made this work possible, and especially to @binxuwang.bsky.social. This work grew from a year of collaboration! Tomorrow, Part II: geometry of concepts and Minkowski Representation Hypothesis. 🕹️ kempnerinstitute.github.io/dinovision 📄 arxiv.org/pdf/2510.08638

000

Thomas Fel @thomasfel.bsky.social · 14/10/2025

Curious tokens, the registers. DINO seems to use them to encode global invariants: we find concepts (directions) that fire exclusively (!) on registers. Example of such concepts include motion blur detector and style (game screenshots, drawings, paintings, warped images...)

100

Thomas Fel @thomasfel.bsky.social · 14/10/2025

Now for depth estimation. How does DINO know depth? It turns out it has discovered several human-like monocular depth cues: texture gradients resembling blurring or bokeh, shadow detectors, and projective cues. Most units mix cues, but a few remain remarkably pure.

100

Thomas Fel @thomasfel.bsky.social · 14/10/2025

Another surprise here: the most important concepts are not object-centric at all, but boundary detectors. Remarkably, these concepts coalesce into a low-dimensional subspace within (see paper).

110

Thomas Fel @thomasfel.bsky.social · 14/10/2025

Let's zoom in on classification. For every class, we find two concepts: one fires on the object (e.g., "rabbit"), and another fires everywhere *except* the object -- but only when it's present! We call them Elsewhere Concepts (credit: @davidbau.bsky.social).

110

Thomas Fel @thomasfel.bsky.social · 14/10/2025

Assuming the Linear Rep. Hypothesis, SAEs arise naturally as instruments for concept extraction, they will be our companions in this descent. Archetypal SAE uncovered 32k concepts. Our first observation: different tasks recruit distinct regions of this conceptual space.

100

Thomas Fel @thomasfel.bsky.social · 14/10/2025

🕳️🐇 𝙄𝙣𝙩𝙤 𝙩𝙝𝙚 𝙍𝙖𝙗𝙗𝙞𝙩 𝙃𝙪𝙡𝙡 – 𝙋𝙖𝙧𝙩 𝙄 (𝑃𝑎𝑟𝑡 𝐼𝐼 𝑡𝑜𝑚𝑜𝑟𝑟𝑜𝑤) 𝗔𝗻 𝗶𝗻𝘁𝗲𝗿𝗽𝗿𝗲𝘁𝗮𝗯𝗶𝗹𝗶𝘁𝘆 𝗱𝗲𝗲𝗽 𝗱𝗶𝘃𝗲 𝗶𝗻𝘁𝗼 𝗗𝗜𝗡𝗢𝘃𝟮, one of vision’s most important foundation models. And today is Part I, buckle up, we're exploring some of its most charming features. :)

23612

Thomas Fel @thomasfel.bsky.social · 17/09/2025

One interesting result: our "Bridge Score" points to concept pairs that connect vision & language. In the demo you can explore these bridges (links) and see how multimodality shows up ! :) with @isabelpapad.bsky.social, @chloesu07.bsky.social, @shamkakade.bsky.social and Stephanie Gil

020

Thomas Fel @thomasfel.bsky.social · 17/09/2025

Check out our COLM 2025 (oral) 🎤 SAEs reveal that VLM embedding spaces aren’t just "image vs. text" cones. They contain stable conceptual directions, some forming surprising bridges across modalities. arxiv.org/abs/2504.11695 Demo 👉 vlm-concept-visualization.com

150

Thomas Fel @thomasfel.bsky.social · 30/01/2025

DinoV2, C:5232... 😶‍🌫️

210

Thomas Fel @thomasfel.bsky.social · 04/12/2024

I’ll be at @neuripsconf.bsky.social this year, sharing some work on explainability and representations. If you’re attending and want to chat, feel free to reach out !👋

0101

Thomas Fel @thomasfel.bsky.social · 27/11/2024

A fun thesis experiment: ResNet, DETR, and CLIP tackle Saint-Bernards. 🐶 ResNet focused on **fur** patterns, DETR too but also use **paws** (possibly because it helps define bounding boxes), and CLIP **head** concept oddly included human heads — language shaping learned concepts?

An image showing how three model top concept look like to classify st bernard, resnet use head and fur, while detr also use paws (maybe it help him delimitate the boundary). Clip use the head of the st bernard, but oodly the head seems to also react to human head...

082