iiinova — where your digital self keeps evolving

EvolutionaryScale

@evolutionaryscale.bsky.social

EvolutionaryScale @evolutionaryscale.bsky.social · 04/12/2024

Information about protein structure improves predictably with increasing training compute, demonstrating linear scaling across multiple orders of magnitude. We overtrained ESM C 300M and 600M beyond the compute-optimal point predicted by the scaling curves.

Scaling curves for ESM C protein language models. Model performance improves predictably with scale.

060

EvolutionaryScale @evolutionaryscale.bsky.social · 04/12/2024

Introducing ESM Cambrian, a new family of protein language models, focused on creating representations of the underlying biology of proteins.

Model Scale vs. Performance curves for ESM C models, with comparisons to ESM2 and other protein LMs. ESMC performs better than existing state of the art for the same model parameter scale.

15016