This is a collaboration with @nkgarg.bsky.social and Jon Kleinberg. See the paper here: arxiv.org/abs/2610.05794
I’m also on the academic job market this year. If you find this work exciting, would love to chat more!
arxiv.org
A Testable Theory of Atomic Features
We develop and test a theory of language model representations in which there exist atomic features. Our main theoretical insight is that in such a model, sparse dictionaries (e.g., SAEs) of increasin...