iiinova — where your digital self keeps evolving

Jesseba Fernando

@jesseba.bsky.social

88 followers 409 following 21 posts

PhD Student @nunetsi.bsky.social Put me in the middle of nowhere with coffee and I'm happy jesseba.github.io

Posts Replies Media

Jesseba Fernando @jesseba.bsky.social · 11/06/2026

Check out the first in a 3-part series of posts explaining @guitchounts.bsky.social and I's latest paper exploring dynamics of transformers! Watching a transformer spin open.substack.com/pub/oftwomin...

Title of article with a background depicting a spiral

011

Jesseba Fernando @jesseba.bsky.social · 03/09/2025

Packed room at our satellite!!

011

Jesseba Fernando @jesseba.bsky.social · 28/05/2025

Super, super excited to be here at the sixth annual conference on the Mathematics of Neuroscience and AI!

010

Jesseba Fernando @jesseba.bsky.social · 21/02/2025

11/ Finally, we asked: Does the RS behave like a dynamical system? To test this, we teleported RS vectors to different positions in PCA space and observed their evolution. Result? Early layers exhibit attractor-like behavior—pushing activations back toward their “natural” trajectory.

110

Jesseba Fernando @jesseba.bsky.social · 21/02/2025

10/ To visualize global RS behavior, we trained a Compressing Autoencoder (CAE)—squashing activations into 2D. The CAE finds low-dimensional population dynamics, which showed that early layers are harder to reconstruct, while later layers become lower-dimensional and more structured.

110

Jesseba Fernando @jesseba.bsky.social · 21/02/2025

9/ Looking at individual unit dynamics, we found rotational trajectories in activation space! Some units spin around a fixed point, others spiral outward. On average, RS units circle ~10 times over 64 layers.

110

Jesseba Fernando @jesseba.bsky.social · 21/02/2025

8/ And here’s something even weirder: mutual information drops in early layers, then slowly climbs back up. Why? We’re not sure—perhaps early layers are quite predictable from one to the next, but in a nonlinear way...

110

Jesseba Fernando @jesseba.bsky.social · 21/02/2025

7/ Zooming out: the cosine similarity of the RS vector between layers increases. Translation: The model’s internal representations become more aligned as we move deeper through the model. But they also accelerate!

110

Jesseba Fernando @jesseba.bsky.social · 21/02/2025

6/ Second surprise: RS activations are highly correlated layer to layer. A given unit’s activation is highly correlated across layers—especially within a layer (attention to MLP) vs. across layers.

110

Jesseba Fernando @jesseba.bsky.social · 21/02/2025

5/ First surprise: RS activations increase in density over layers. Early layers are sparse, later layers are dense. Why? There’s no a priori reason for this—blocks could just as easily write balancing negative values. But they don’t.

110

Jesseba Fernando @jesseba.bsky.social · 21/02/2025

2/ @guitchounts.bsky.social and I went on a little side-quest in which we took a page out of neuroscience to see if we can understand AI—LLMs in this case—a little better. 🧵

110