Reposted by Matteo Pagliardini
new extensive evaluation of different optimizers for LLM training
arxiv.org/abs/2509.01440
arxiv.org
Benchmarking Optimizers for Large Language Model Pretraining
The recent development of Large Language Models (LLMs) has been accompanied by an effervescence of novel ideas and methods to better optimize the loss of deep learning models. Claims from those method...
