Quiz time: I shrank my pretrained LLM's VRAM usage from 1TB+ to 50GB with zero drop in performance. What did I do?
1) 🪄Quantization🪄
2) ⚡FlashAttention⚡
3) 🏁Gradient checkpointing🏁
4) 💾KV cache to CPU💾
J Rosser
@jrosseruk.bsky.social
19 followers 71 following 5 posts
Research Scientist Intern at Spotify | DPhil in Machine Learning at the University of Oxford
