This is a fascinating deep dive by Welch Labs into how LLMs "grok", meaning to move beyond just memorization to generalization after continue to train for long time while seemingly making to progress.
Turns how waves, trigonometry and identity functions are everywhere. Highly recommended. #MLSky
youtube.com
The most complex model we actually understand
YouTube video by Welch Labs