Sign in

Etowah Adams

@etowah0.bsky.social
106 followers 228 following 31 posts

enjoying and bemoaning biology. phd student @columbia prev. @harvardmed @ginkgo @yale

PostsRepliesMedia
Etowah Adams @etowah0.bsky.social · 21/08/2026
Additional thanks to @moalquraishi.bsky.social, Fergus Imrie, Frank von Delft, @jchodera.bsky.social, Charlotte Deane, and the people who believed in OpenBind to get it initially funded: Charlie Harris, Matt Clifford, Andrew Crawford, @tomwestgarth.bsky.social
000 0
Etowah Adams @etowah0.bsky.social · 21/08/2026
Huge thanks to the entire OpenBind team and its funders, @asapdiscovery.bsky.social , Syngenta, and @diamondlightsource.bsky.social , and to everyone who helped generate, process, and benchmark these data.
100 0
Etowah Adams @etowah0.bsky.social · 21/08/2026
Read more on our blog post: openbind.uk/news/blog-op... And try the model out: github.com/aqlaboratory...
openbind.uk
Blog: OpenBind-0: Advancing Open Molecular Structure Prediction | openbind.uk
Introducing OpenBind-0, a fully open molecular structure prediction model. This post presents the model, its performance, and new benchmark data that highli ...
100 0
Etowah Adams @etowah0.bsky.social · 21/08/2026
We hope OB0 will be a useful model for the community to use and build on. As OpenBind generates more data, OB0 will serve as a baseline for measuring its impact and help guide what data we collect next.
100 0
Etowah Adams @etowah0.bsky.social · 21/08/2026
And these are hard drug targets! On Dengue and Zika RdRp, no cofolding model exceeded 10% top-25 success (best: 7.7% and 6.7%). The target systems combine flexible binding sites, multiple pockets, and poor training-set coverage.
100 0
Etowah Adams @etowah0.bsky.social · 21/08/2026
We’re also releasing 717 discovery-relevant ligand-bound structures across three targets: FatA, Dengue RdRp, and Zika RdRp. We believe that releasing hard, real-world evaluation sets alongside models is a good practice moving forward.
100 0
Etowah Adams @etowah0.bsky.social · 21/08/2026
Pre-training on new data may provide benefits beyond fine-tuning. OB0, which saw the fragment structures during pre-training, outperforms a version of OF3p2 fine-tuned on those fragments when predicting follow-on compounds.
100 0
Etowah Adams @etowah0.bsky.social · 21/08/2026
How does OB0 do in a fragment-based drug discovery context? Bound fragment structures guide follow-on design. We previously showed that fine-tuning on fragment structures improves follow-on predictions. What if they’re seen during pre-training?
100 0
Etowah Adams @etowah0.bsky.social · 21/08/2026
This highlights an important distinction. More data doesn't necessarily translate into improve models. Models improve most when new data fills gaps in the training set and provides relevant examples they didn’t have before.
100 0
Etowah Adams @etowah0.bsky.social · 21/08/2026
The biggest gains come where new data adds more relevant training examples: test complexes that become more similar to the 2025 training set see substantial improvements in accuracy. Those that stay in the same similarity bin show little difference.
100 0
Etowah Adams @etowah0.bsky.social · 21/08/2026
Do four more years of training data make the test set easier? Less than you might think. Most test complexes are about as similar to the 2021 training set as they are to the 2025 training set.
100 0
Etowah Adams @etowah0.bsky.social · 21/08/2026
OB0 is the first in a series of OpenBind models, with each release incorporating newer PDB structures and OpenBind data. It sets a baseline for measuring the impact of OpenBind data. Future models will add further new small-molecule capabilities.
100 0
Etowah Adams @etowah0.bsky.social · 21/08/2026
Unlike OpenFold3-preview2 and AF3, trained on data through 2021, OpenBind-0 (OB0) uses data through June 2025 and the forthcoming OpenFold3 architecture. It is competitive with leading cofolding models and adds chemical steering to improve ligand validity.
100 0
Etowah Adams @etowah0.bsky.social · 21/08/2026
OpenBind intends to collect 10,000s of protein-ligand structures & affinities. To prioritize what we collect next, we need cofolding models trained on the latest data. Today we're releasing OpenBind-0 and 717 new ligand-bound structures.
1154 0
Reposted by Etowah Adams
Reciprocal Space Station @rs-station.bsky.social · 21/07/2026
1/ We're excited to announce that @rs-station.bsky.social is joining the Open Molecular Software Foundation @omsf.io!
1105 0
Etowah Adams @etowah0.bsky.social · 13/03/2026
Huge resource for the community. Turning compute into data to train models to turn more compute into data...the cycle must go on bsky.app/profile/moal...
040 0
Reposted by Etowah Adams
Mohammed AlQuraishi @moalquraishi.bsky.social · 13/03/2026
New OpenFold3 preview out! (OF3p2) It closes the gap to AlphaFold3 for most modalities. Most critically, we're releasing everything, including training sets & configs, making OF3p2 the only current AF3-based model that is functionally trainable & reproducible from scratch🧵1/9
124590 0
Reposted by Etowah Adams
Tristan Bepler @tbepler.bsky.social · 11/02/2025
Excited to share PoET-2, our next breakthrough in protein language modeling. It represents a fundamental shift in how AI learns from evolutionary sequences. 🧵 1/13
13215 0
Reposted by Etowah Adams
Mohammed AlQuraishi @moalquraishi.bsky.social · 10/02/2025
It's long seemed that molecular biology is a natural home for ML interpretability research, given the maturity of human-constructed models of biological mechanisms—permitting direct comparison with their ML-derived counterparts—unlike vision and NLP. Our first foray below👇.
03815 0
Reposted by Etowah Adams
Diego del Alamo @delalamo.xyz · 10/02/2025
This might be the best paper on applying sparse autoencoders to protein language models. The authors identify how neural networks trained on amino acid sequences "discover" different features, some specific to individual protein families, other for substructures www.biorxiv.org/content/10.1...
A range of features identified from sparse autoencoders trained on different layers on ESM2-650M
24411 0
Etowah Adams @etowah0.bsky.social · 10/02/2025
Thank you for your help interpreting features :) It was really special having the community squint at these features with us. Lots of weird features means that everyone sees different/new things!
020 0
Etowah Adams @etowah0.bsky.social · 10/02/2025
Thanks to our coauthors Minji Lee, Steven Yu, and @moalquraishi.bsky.social! Check work from Elana Pearl on using SAEs on pLMs too! x.com/ElanaPearl/s...
000 0
Etowah Adams @etowah0.bsky.social · 10/02/2025
We’re excited about the potential of SAEs in biology and would love to hear your ideas. Our preprint: www.biorxiv.org/content/10.1... Visualizer: interprot.com Github: github.com/etowahadams/... HuggingFace: huggingface.co/liambai/Inte...
biorxiv.org
From Mechanistic Interpretability to Mechanistic Biology: Training, Evaluating, and Interpreting Sparse Autoencoders on Protein Language Models
Protein language models (pLMs) are powerful predictors of protein structure and function, learning through unsupervised training on millions of protein sequences. pLMs are thought to capture common mo...
121 0
Etowah Adams @etowah0.bsky.social · 10/02/2025
Such hard-to-interpret but predictive features could result from biases and limitations in our datasets and models. However, there is another intriguing possibility: these features may correspond to biological mechanisms that have yet to be discovered.
100 0
Etowah Adams @etowah0.bsky.social · 10/02/2025
Not all predictive features are easily interpretable. For instance, a predictor for membrane localization, latent L28/3154, predominantly activates on poly-alanine sequences, whose functional relevance remains unclear.
100 0
Etowah Adams @etowah0.bsky.social · 10/02/2025
When trained to predict thermostability, the linear models assign their most positive weights to features that correlate with hydrophobic amino acids and their most negative weights to a feature activating on glutamate, an amino acid linked to instability.
100 0
Etowah Adams @etowah0.bsky.social · 10/02/2025
For the extracellular class, we uncover features that activate on signal peptides, which trigger the cellular mechanisms that transport proteins outside the cell.
100 0
Etowah Adams @etowah0.bsky.social · 10/02/2025
These sequence motifs consist of K/R rich regions and act like a “passport” for proteins to enter the nucleus. As many NLS variants still remain unknown, our SAE-based approach has potential to become a novel method to aid their discovery and characterization.
100 0
Etowah Adams @etowah0.bsky.social · 10/02/2025
For example, we trained a linear classifier on subcellular localization labels (nucleus, cytoplasm, mitochondrion…). Excitingly, the most highly weighted features for the nucleus class recognize nuclear localization signals (NLS)
120 0
Etowah Adams @etowah0.bsky.social · 10/02/2025
Having established methods to interpret features, we sought to associate features with functional properties and more generally understand pLM performance on downstream tasks. We train linear models on SAE features and analyze features that contribute the most to task performance.
100 0
Etowah Adams @etowah0.bsky.social · 10/02/2025
We categorize SAE latents by family specificity and activation pattern by layer. We find more family specific features in middle layers, and more single token activating (point) features in layer layers. We also study how SAE hyperparameters change what features are uncovered.
100 0
Etowah Adams @etowah0.bsky.social · 10/02/2025
We find many features appear highly specific to certain protein families, suggesting pLMs contain an internal notion of protein families which mirrors known homology-based families. Other features correspond to broader, family-independent coevolutionary signals.
120 0
Etowah Adams @etowah0.bsky.social · 10/02/2025
(Regarding the recent discourse surrounding SAEs, we refer to this thread from @banburismus_) x.com/banburismus_...
100 0
Etowah Adams @etowah0.bsky.social · 10/02/2025
Using sparse autoencoders (SAEs), we find many interpretable features in a protein language model (pLM), ESM-2, ranging from secondary structure, to conserved domains, to context specific properties. Explore them at interprot.com!
121 0
Etowah Adams @etowah0.bsky.social · 10/02/2025
For more background on mechanistic interpretability and protein language models, check out this previous thread on our work (X link, sorry) x.com/liambai21/st...
110 0
Etowah Adams @etowah0.bsky.social · 10/02/2025
Can we learn protein biology from a language model? In new work led by @liambai.bsky.social and me, we explore how sparse autoencoders can help us understand biology—going from mechanistic interpretability to mechanistic biology.
24524 0