After finishing my papers for my PhD, I spent some time exploring new directions. I ended up working on Diffusion Language Models with @haoyuhe.bsky.social (he made it work 🚀), @yongcao.bsky.social, @andreasgeiger.bsky.social.
I learned a lot of new things and I am very excited about the results. 🥳
arxiv.org
MDPO: Overcoming the Training-Inference Divide of Masked Diffusion Language Models
Diffusion language models, as a promising alternative to traditional autoregressive (AR) models, enable faster generation and richer conditioning on bidirectional context. However, they suffer from a ...
