iiinova — where your digital self keeps evolving

Reposted by Austin Wang

Anshul Kundaje @anshulkundaje.bsky.social · 19/06/2025

@saramostafavi.bsky.social (@Genentech) & I (@Stanford) r excited to announce co-advised postdoc positions for candidates with deep expertise in ML for bio (especially sequence to function models, causal perturbational models & single cell models). See details below. Pls RT 1/

15439

Reposted by Austin Wang

Anshul Kundaje @anshulkundaje.bsky.social · 15/05/2025

Today was a big day for the lab. We had two back to back thesis defenses and the defenders defended with great science and character. Congrats to DR. Kelly Cochran & DR. @soumyakundu.bsky.social on this momentous achievement. Brilliant scientists with brilliant futures ahead. 🎉🎉🎉

2777

Reposted by Austin Wang

Selin Jessa @selinjessa.com · 03/05/2025

Delighted to share our latest work deciphering the landscape of chromatin accessibility and modeling the DNA sequence syntax rules underlying gene regulation during human fetal development! www.biorxiv.org/content/10.1... Read on for more: 🧵 1/16 #GeneReg 🧬🖥️

biorxiv.org

Dissecting regulatory syntax in human development with scalable multiomics and deep learning

Transcription factors (TFs) establish cell identity during development by binding regulatory DNA in a sequence-specific manner, often promoting local chromatin accessibility, and regulating gene expre...

212860

Reposted by Austin Wang

Jacob Schreiber @jmschreiber91.bsky.social · 24/04/2025

Our preprint on designing and editing cis-regulatory elements using Ledidi is out! Ledidi turns *any* ML model (or set of models) into a designer of edits to DNA sequences that induce desired characteristics. Preprint: www.biorxiv.org/content/10.1... GitHub: github.com/jmschrei/led...

biorxiv.org

Programmatic design and editing of cis-regulatory elements

The development of modern genome editing tools has enabled researchers to make such edits with high precision but has left unsolved the problem of designing these edits. As a solution, we propose Ledi...

211437

Reposted by Austin Wang

Anshul Kundaje @anshulkundaje.bsky.social · 07/01/2025

Very excited to announce that the single cell/nuc. RNA/ATAC/multi-ome resource from ENCODE4 is now officially public. This includes raw data, processed data, annotations and pseudobulk products. Covers many human & mouse tissues. 1/ www.encodeproject.org/single-cell/...

encodeproject.org

Single cell – ENCODEHomo sapiens clickable body map

628786

Austin Wang @austintwang.bsky.social · 14/12/2024

I think that’ll be interesting to look more into! The profile information does not convey overall accessibility since it’s normalized, but maybe this sort of multitasking could help.

010

Austin Wang @austintwang.bsky.social · 11/12/2024

Thank you for the kind words! Yes, ChromBPNet uses unmodified models, which includes profile data and a bias model. However these evaluations use only the count head.

110

Reposted by Austin Wang

Arpita Singhal @arpita-s.bsky.social · 11/12/2024

Excited to announce DART-Eval, our latest work on benchmarking DNALMs! Catch us at #NeurIPS!

085

Reposted by Austin Wang

amanpatel100.bsky.social @amanpatel100.bsky.social · 11/12/2024

New work! Come check out our poster tomorrow and take a look at the paper!

063

Austin Wang @austintwang.bsky.social · 11/12/2024

(9/10) How do we train more effective DNALMs? Use better data and objectives: • Nailing short-context tasks before long-context • Data sampling to account for class imbalance • Conditioning on cell type context These strategies use external annotations, which are plentiful!

171

Austin Wang @austintwang.bsky.social · 11/12/2024

(8/10) This indicates that DNALMs inconsistently learn functional DNA. We believe that the culprit is not architecture, but rather the sparse and imbalanced distribution of functional DNA elements. Given their resource requirements, current DNALMs are a hard sell.

171

Austin Wang @austintwang.bsky.social · 11/12/2024

(7/10) DNALMs struggle with more difficult tasks. Furthermore, small models trained from scratch (<10M params) routinely outperform much larger DNALMs (>1B params), even after LoRA fine-tuning! Our results on the hardest task - counterfactual variant effect prediction.

361

Austin Wang @austintwang.bsky.social · 11/12/2024

(6/10) We introduce DART-Eval, a suite of five biologically informed DNALM evaluations focusing on transcriptional regulatory DNA ordered by increasing difficulty.

141

Austin Wang @austintwang.bsky.social · 11/12/2024

(5/10) Rigorous evaluations of DNALMs, though critical, are lacking. Existing benchmarks: • Focus on surrogate tasks tenuously related to practical use cases • Suffer from inadequate controls and other dataset design flaws • Compare against outdated or inappropriate baselines

120

Austin Wang @austintwang.bsky.social · 11/12/2024

(4/10) An effective DNALM should: • Learn representations that can accurately distinguish different types of functional DNA elements • Serve as a foundation for downstream supervised models • Outperform models trained from scratch

120

Austin Wang @austintwang.bsky.social · 11/12/2024

(3/10) However, DNA is vastly different from text, being much more heterogeneous, imbalanced, and sparse. Imagine a blend of several different languages interspersed with a load of gibberish.

183

Austin Wang @austintwang.bsky.social · 11/12/2024

(2/10) DNALMs are a new class of self-supervised models for DNA, inspired by the success of LLMs. These DNALMs are often pre-trained solely on genomic DNA without considering any external annotations.

130