iiinova — where your digital self keeps evolving

Takeaway: LLM training exhibits multi-phasic information geometry changes! ✨ - Pretraining: Compress → Expand (Memorize) → Compress (Generalize). - Post-training: SFT/DPO → Expand; RLVR → Consolidate. Representation geometry offers insights into when models memorize vs. generalize! 🤓 🧵8/9

The multi-phasic information geometry changes in LLM pretraining and post-training. Pretraining undergoes an initial warmup phase, which corresponds to echolalia behavior, followed by entropy-seeking where the model learns high-frequency n-gram statistics, and finally a compression-seeking phase, where the model learns long-range dependencies. The post-training stages of SFT and DPO exhibit entropy-seeking behavior, where the model memorizes instruction following behavior, whereas RLVR exhibits compress-seeking behavior, where the model learns generalized reasoning at the cost of exploration.

130

BONUS: Is task-relevant info contained in the top eigendirections? On SciQ: - Removing top 10/50 directions barely hurts accuracy.✅ - Retaining only top 10/50 directions CRUSHES accuracy.📉 As supported by our theoretical results, eigenspectrum tail encodes critical task information! 🤯 🧵7/9

Why do these geometric phases arise?🤔 We show, both through theory and with simulations in a toy model, that these non-monotonic spectral changes occur due to gradient descent dynamics with cross-entropy loss under 2 conditions: 1. skewed token frequencies 2. representation bottlenecks 🧵6/9

140

Post-training also yields distinct geometric signatures: - SFT & DPO exhibit entropy-seeking expansion, favoring instruction memorization but reducing OOD robustness.📈 - RLVR exhibits compression-seeking consolidation, learning reward-aligned behaviors at the cost of reduced exploration.📉 🧵5/9

Supervised Finetuning (SFT) exhibits entropy-seeking expansion, coupled with decreased OOD robustness, whereas Reinforcement Learning from Verifiable Rewards (RLVR) exhibits compression-seeking consolidation, coupled with reduced exploration.

How do these phases relate to LLM behavior? - Entropy-seeking: Correlates with short-sequence memorization (♾️-gram alignment). - Compression-seeking: Correlates with dramatic gains in long-context factual reasoning, e.g. TriviaQA. Curious about ♾️-grams? See: bsky.app/profile/liuj... 🧵4/9

Different geometric phases correspond to acquiring different behaviors: entropy-seeking phase correlates with increased short-sequence memorization, compression-seeking phase correlates with better TriviaQA performance.

150

LLMs have 3 pretraining phases: Warmup: Rapid compression, collapsing representation to dominant directions. Entropy-seeking: Manifold expansion, adding info in non-dominant directions.📈 Compression-seeking: Anisotropic consolidation, selectively packing more info in dominant directions.📉 🧵3/9

OLMo-2 and Pythia models undergo multiple distinct geometric phases during pretraining, indicating a non-monotonic change in representation complexity underlying monotonic decrease in training loss.

160

When investigating OLMo (@ai2.bsky.social) & Pythia (@eleutherai.bsky.social) model checkpoints, as expected, pretraining loss ⬇️monotonically. BUT 🎢The spectral metrics (RankMe, αReQ) change non-monotonically (with more pretraining)! Takeaway: We discover geometric phases of LLM learning! 🧵2/9

150

📐We measured representation complexity using the #eigenspectrum of the final layer representations. We used 2 spectral metrics: - Spectral Decay Rate, αReQ: Fraction of variance in non-dominant directions. - RankMe: Effective Rank; #dims truly active. ⬇️αReQ ⇒ ⬆️RankMe ⇒ More complex! 🧵1/9

Spectral decomposition methods and metrics used to quantify representation space complexity in LLMs.

170

Arna Ghosh @arnaghosh.bsky.social · 19/09/2025

LLMs are trained to compress data by mapping sequences to high-dim representations! How does the complexity of this mapping change across LLM training? How does it relate to the model’s capabilities? 🤔 Announcing our #NeurIPS2025 📄 that dives into this. 🧵below #AIResearch #MachineLearning #LLM

New paper titled "Tracing the Representation Geometry of Language Models from Pretraining to Post-training" by Melody Z Li, Kumar K Agrawal, Arna Ghosh, Komal K Teru, Adam Santoro, Guillaume Lajoie, Blake A Richards.

16112

Very cool study, with interesting insights about theta sequences and learning!

Blake Richards @tyrellturing.bsky.social · 09/07/2025

Reposted by Arna Ghosh

Together with @repromancer.bsky.social, I have been musing for a while that the exponentiated gradient algorithm we've advocated for comp neuro would work well with low-precision ANNs. This group got it working! arxiv.org/abs/2506.17768 May be a great way to reduce AI energy use!!! #MLSky 🧪

arxiv.org

Log-Normal Multiplicative Dynamics for Stable Low-Precision Training of Large Networks

Studies in neuroscience have shown that biological synapses follow a log-normal distribution whose transitioning can be explained by noisy multiplicative dynamics. Biological networks can function sta...

33913

Arna Ghosh @arnaghosh.bsky.social · 16/06/2025

This looks like a very cool result! 😀 Can't wait to read in detail.

030

Reposted by Arna Ghosh

Yiğit Demirağ @yigit.ai · 08/05/2025

Our team is hiring in Zurich! www.google.com/about/career...

google.com

Research Scientist, Paradigms of Intelligence — Google Careers

0153

Arna Ghosh @arnaghosh.bsky.social · 02/04/2025

Re diff implicit biases of architecture: the metrics implemented here (roughly) characterize the eigenspectrum (eigenval distribution) of the representation space. They don't really incorporate the eigenvector information --> hence, "what" features don't matter, only "how" matters.

000

Arna Ghosh @arnaghosh.bsky.social · 02/04/2025

Comparing models of different architectures often is tricky because of the different implicit biases of each architecture. RSA can be helpful to in some cases. But if you are looking for a metric (number) that tells you which model is better, I have some ideas but they are not implemented here. 😜

100

Indeed! The metrics work best when comparing networks of comparable architectures, though. So, if you are looking to select the best model checkpoint from a pretraining routine or diff hyperparam configurations, and the loss function is not as insightful, these metrics are incredibly helpful. :)

Also, big shoutout to @quentin-garrido.bsky.social+gang and @aggieinca.bsky.social+gang for developing Rankme and Lidar, respectively. Reptrix incorporates these representation quality metrics. 🚀 Let's make it easier to select good SSL/foundation models. 💪

020

PS: It was fun to put Reptrix together with @daniebenes.bsky.social, our open-source expert and new addition to the α-Squad comprising of Arnab Mondal, @kumarkagrawal.bsky.social @tyrellturing.bsky.social and I. [6/6]

030

reptrix/CONTRIBUTING.md at main · BARL-SSL/reptrix

If you do end up giving Reptrix a try, we would love to hear from you! We also encourage you to contribute to our repo and add example notebooks trying out these metrics for your networks, beyond natural language and vision domains. github.com/BARL-SSL/rep... [5/6]

github.com

Library that provides metrics to assess representation quality - BARL-SSL/reptrix

Metrics for evaluating representation quality: - α-ReQ: Measures discriminativeness. Lower alpha = better! - RankMe: Assesses representation capacity. Higher rank = higher capacity! - LiDAR: Evaluates separability among object manifolds. Higher rank = better separability! [4/6]

✨ Key Features of Reptrix: - 📈 Suite of metrics to assess representation quality: α-ReQ, RankMe, LiDAR, and more! - 🤝 Seamless PyTorch integration for minimal setup, maximum insights - 💻 Open Source: Contribute and enhance! [3/6]

α-ReQ: Assessing Representation Quality in SSL | Mila

Inspired by conversations after our α-ReQ paper (NeurIPS 2022) and subsequent work, we created Reptrix as an open-source library for assessing representation quality across models of vision, language… and more. Check out our @mila-quebec.bsky.social blogpost: mila.quebec/en/article/a... [2/6]

mila.quebec

The success of self-supervised learning algorithms has drastically changed the landscape of training deep neural networks. With well-engineered architectures and training objectives, SSL models learn ...