iiinova — where your digital self keeps evolving

Beatrix M. G. Nielsen

@beatrixmgn.bsky.social

PhD student in machine learning at DTU, Copenhagen. Especially interested in model representations.

Beatrix M. G. Nielsen @beatrixmgn.bsky.social · 21/10/2025

2/5 We prove that a small KL divergence between models is not enough to guarantee similar representations. Here is an example of how to construct two models with small KL divergence, but representations which are far from being linear transformations of each other.

110

Beatrix M. G. Nielsen @beatrixmgn.bsky.social · 01/08/2025

Thank you to everyone who came to talk to us at ACL! Very happy that so many people are interested in our work :D If you didn’t manage to have a look yet: This work is relevant if you want to compare representations in language models, and it can be read here: arxiv.org/abs/2502.10201

040

Beatrix M. G. Nielsen @beatrixmgn.bsky.social · 07/07/2025

Our paper "Prediction Hubs are Context-Informed Frequent Tokens in LLMs" has been accepted at ACL 2025! Main points: 1. Hubness is not a problem when language models do next-token prediction. 2. Nuisance hubness can appear when other comparisons are made.

181