iiinova — where your digital self keeps evolving

Takeaway: LLM training exhibits multi-phasic information geometry changes! ✨ - Pretraining: Compress → Expand (Memorize) → Compress (Generalize). - Post-training: SFT/DPO → Expand; RLVR → Consolidate. Representation geometry offers insights into when models memorize vs. generalize! 🤓 🧵8/9

The multi-phasic information geometry changes in LLM pretraining and post-training. Pretraining undergoes an initial warmup phase, which corresponds to echolalia behavior, followed by entropy-seeking where the model learns high-frequency n-gram statistics, and finally a compression-seeking phase, where the model learns long-range dependencies. The post-training stages of SFT and DPO exhibit entropy-seeking behavior, where the model memorizes instruction following behavior, whereas RLVR exhibits compress-seeking behavior, where the model learns generalized reasoning at the cost of exploration.

130

Why do these geometric phases arise?🤔 We show, both through theory and with simulations in a toy model, that these non-monotonic spectral changes occur due to gradient descent dynamics with cross-entropy loss under 2 conditions: 1. skewed token frequencies 2. representation bottlenecks 🧵6/9

140

Post-training also yields distinct geometric signatures: - SFT & DPO exhibit entropy-seeking expansion, favoring instruction memorization but reducing OOD robustness.📈 - RLVR exhibits compression-seeking consolidation, learning reward-aligned behaviors at the cost of reduced exploration.📉 🧵5/9

Supervised Finetuning (SFT) exhibits entropy-seeking expansion, coupled with decreased OOD robustness, whereas Reinforcement Learning from Verifiable Rewards (RLVR) exhibits compression-seeking consolidation, coupled with reduced exploration.

120

How do these phases relate to LLM behavior? - Entropy-seeking: Correlates with short-sequence memorization (♾️-gram alignment). - Compression-seeking: Correlates with dramatic gains in long-context factual reasoning, e.g. TriviaQA. Curious about ♾️-grams? See: bsky.app/profile/liuj... 🧵4/9

Different geometric phases correspond to acquiring different behaviors: entropy-seeking phase correlates with increased short-sequence memorization, compression-seeking phase correlates with better TriviaQA performance.

150

LLMs have 3 pretraining phases: Warmup: Rapid compression, collapsing representation to dominant directions. Entropy-seeking: Manifold expansion, adding info in non-dominant directions.📈 Compression-seeking: Anisotropic consolidation, selectively packing more info in dominant directions.📉 🧵3/9

OLMo-2 and Pythia models undergo multiple distinct geometric phases during pretraining, indicating a non-monotonic change in representation complexity underlying monotonic decrease in training loss.

160

📐We measured representation complexity using the #eigenspectrum of the final layer representations. We used 2 spectral metrics: - Spectral Decay Rate, αReQ: Fraction of variance in non-dominant directions. - RankMe: Effective Rank; #dims truly active. ⬇️αReQ ⇒ ⬆️RankMe ⇒ More complex! 🧵1/9

Spectral decomposition methods and metrics used to quantify representation space complexity in LLMs.

170

Arna Ghosh @arnaghosh.bsky.social · 01/04/2025

LLMs are trained to compress data by mapping sequences to high-dim representations! How does the complexity of this mapping change across LLM training? How does it relate to the model’s capabilities? 🤔 Announcing our #NeurIPS2025 📄 that dives into this. 🧵below #AIResearch #MachineLearning #LLM

New paper titled "Tracing the Representation Geometry of Language Models from Pretraining to Post-training" by Melody Z Li, Kumar K Agrawal, Arna Ghosh, Komal K Teru, Adam Santoro, Guillaume Lajoie, Blake A Richards.

16112

Are you training self-supervised/foundation models, and worried if they are learning good representations? We got you covered! 💪 🦖Introducing Reptrix, a #Python library to evaluate representation quality metrics for neural nets: github.com/BARL-SSL/rep... 🧵👇[1/6] #DeepLearning

3279

Arna Ghosh @arnaghosh.bsky.social · 10/02/2025

Just over a week since I defended my 🤖+🧠PhD thesis, and the feeling is just sinking in. Extremely grateful to @tyrellturing.bsky.social for supporting me through this amazing journey! 🙏 Big thanks to all members of the LiNC lab, and colleagues at mcgill University and @mila-quebec.bsky.social. ❤️😁

3303

🧵 7/8 Combining smaller projectors, higher # augmentations, and optimal orthogonalization advances the compute-performance Pareto frontier in SSL: 50% less compute for the same downstream performance or better accuracy at a fixed compute budget. 📈

🧵 6/8 Multiple augmentations lead to better approximation of the underlying data similarity kernel. More views → better kernel estimation → improved convergence and sample efficiency, matching larger datasets in low-data settings. 4 views 50% data 🤝 2 views 100% data. 📊

🧵 5/8 Surprising result: Smaller projector dimensions (reducing projector params by up to 100x) + stronger orthogonalization can match the performance of high-dim projectors! We can get the same representation quality while using less compute 🔥.

🧵 4/8 First key finding: Gradient descent's implicit bias reveals a sweet spot in feature learning. Too little orthogonalization → feature collapse. Too much → unstable learning dynamics.We characterized this trade-off for harnessing the value of small projectors 🎯

120

🧵2/8 While SSL algos like SimCLR/VICReg use diff losses, they optimize the same objective: match the similarity struct given by augmentations. Most are limited to 2 views & need massive compute. We derive an equivalent but computationally efficient formulation of this loss. 🔍