Sign in

Vincent Sitzmann

@vincentsitzmann.bsky.social
1.3K followers 81 following 17 posts

Teaching AI to model, see, and interact with our world. Assistant Professor @ MIT, leading the Scene Representation Group (scenerepresentations.org).

PostsRepliesMedia
Vincent Sitzmann @vincentsitzmann.bsky.social · 11/02/2025
We show that DFoT alone is already a competitive model, matching or beating industry SOTA with way more compute than us. Together with HG, it can stably rollout very long videos, stay robust to out-of-distribution context, and stitch sub-trajectories (6/7)
120
Vincent Sitzmann @vincentsitzmann.bsky.social · 11/02/2025
DFoT enables History Guidance (HG), a family of history-conditioned guidance methods that composes diffusion scores from different histories. From its simplest form to its most advanced variant, HG significantly enhances video diffusion and unlocks new abilities. (5/7)
120
Vincent Sitzmann @vincentsitzmann.bsky.social · 11/02/2025
Unlike previous methods, DFoT views history or target alike as tokens of different noise levels. DFoT trains diffusion with varying noise levels per frame. To conditionally sample, one simply masks out a portion of history with noise before computing the diffusion score. (4/7)
100
Vincent Sitzmann @vincentsitzmann.bsky.social · 11/02/2025
Can we train a single model to perform conditional diffusion with different portions of history - variable lengths, subsets of frames, and even different image-domain frequencies? Introducing DFoT, a simple yet flexible add-on that requires no architectural changes. (3/7)
100
Vincent Sitzmann @vincentsitzmann.bsky.social · 11/02/2025
Classifier-free Guidance (CFG) has been widely used by video diffusion models to boost sample quality. However, researchers rarely perform CFG beyond the first frame. Our paper finds that an equally important conditioning variable, the history, is the long-ignored key. (2/7)
100
Vincent Sitzmann @vincentsitzmann.bsky.social · 11/02/2025
Announcing Diffusion Forcing Transformer (DFoT), our new video diffusion algorithm that generates ultra-long videos of 800+ frames. DFoT enables History Guidance, a simple add-on to any existing video diffusion models for a quality boost. Website: boyuan.space/history-guidance (1/7)
1356