Protein language models don't scale reliably: bigger doesn't always mean better for embeddings. What if what you need from ESM-15B is already inside ESM-8M?
Introducing Reverse Distillation, which flips the usual distillation script: small models anchor and decompose large ones
