Reposted by ngrok
Highly recommended blog and video by @anniesexton.com about the relation between compression algorithms and large language model token prediction.
-> ngrok.com/blog/compres...
ngrok.com
Compression is prediction | ngrok blog
Compression and LLMs are trying to solve the exact same problem: predicting what comes next. Learn the fundamentals of compression and how better prediction leads to better shrinkage.