Reposted by Vernier Fois
One of the issues of being reductionist about LLMs such as calling it a stochastic parrot or token predictor is that otherwise simple systems can have complex emergent properties. For instance, researchers found key weights that act analogously to dopamine neurons.
arxiv.org/abs/2602.00986
arxiv.org
Sparse Reward Subsystem in Large Language Models
Recent studies show that LLM hidden states encode reward-related information, such as answer correctness and model confidence. However, existing approaches typically fit black-box probes on the full h...
