iiinova — where your digital self keeps evolving

Tal Daniel

@taldaniel.bsky.social

Postdoc @ CMU Robotics Institute PhD from the Technion ECE Reseach interests include Unsupervised Representation Learning, Generative Modeling, RL and Robotics. taldatech.github.io

Posts Replies Media

Tal Daniel @taldaniel.bsky.social · 19/02/2025

We believe that leveraging EC-Diffuser’s state generation capability for planning is a promising avenue for future work. This is a joint work with an amazing team: Carl Qi, Dan Haramati, Tal Daniel, Aviv Tamar, and Amy Zhang.

000

Tal Daniel @taldaniel.bsky.social · 19/02/2025

We also trained EC-Diffuser on the real world Language-Table dataset and showed it can also produce high-quality real world rollouts. This demonstrates that the model implicitly matches objects and enforces object consistency over time, aiding in predicting multi-object dynamics.

100

Tal Daniel @taldaniel.bsky.social · 19/02/2025

The result? EC-Diffuser outperforms baselines and achieves zero-shot generalization to novel object configurations—even scaling to more objects than seen during training. See more of the rollouts: sites.google.com/view/ec-diff...

100

Tal Daniel @taldaniel.bsky.social · 19/02/2025

It also enables the Transformer to denoise unordered object-centric particles and actions jointly, capturing multi-modal behavior distributions and complex inter-object dynamics.

100

Tal Daniel @taldaniel.bsky.social · 19/02/2025

Why diffusion? Since noise is added independently to each particle, a simple L1 loss is effective for particle-wise denoising—eliminating the need for complex set-based metrics.

100

Tal Daniel @taldaniel.bsky.social · 19/02/2025

Our model takes in a sequence of unordered state particles (from multi-view images) and actions. Conditioned on the current state and goal, it generates a denoised sequence of future states and actions that can be used for MPC-style control—by executing the first action.

100

Tal Daniel @taldaniel.bsky.social · 19/02/2025

We encode actions as a separate particle. This design allows our Transformer to treat actions and state particles in the same embedding space. We further condition the Transformer with the diffusion timestep and the action tokens via Adaptive layer normalization (AdaLN).

100

Tal Daniel @taldaniel.bsky.social · 19/02/2025

Our entity-centric Transformer is designed to process these unordered particle inputs with a permutation-equivariant architecture, computing self-attention over object-level features without positional embeddings.

100

Tal Daniel @taldaniel.bsky.social · 19/02/2025

We begin by converting high-dimensional pixels into unsupervised object-centric representations using Deep Latent Particles (DLP). Each image is decomposed into an unordered set of latent “particles” from multiple views, capturing key object properties.

100

Tal Daniel @taldaniel.bsky.social · 19/02/2025

This work was led by Carl Qi! Object manipulation from pixels is challenging: high-dimensional, unstructured data creates a combinatorial explosion in states & goals, making multi-object control hard. Traditional BC methods need massive data/compute and still miss the diverse behaviors required.

100

Tal Daniel @taldaniel.bsky.social · 19/02/2025

Check out our new #ICLR2025 paper: EC-Diffuser leverages a novel Transformer-based diffusion denoiser to learn goal-conditioned multi-object manipulation policy from pixels!👇 Paper: www.arxiv.org/abs/2412.18907 Project page: sites.google.com/view/ec-diff... Code: github.com/carl-qi/EC-D...

121

Reposted by Tal Daniel

Mirco Mutti @mircomutti.bsky.social · 13/12/2024

If interested on our take on addressing inverse RL in large state spaces, go to meet @filippo_lazzati and @alberto_metelli in the poster session 5 #NeurIPS2024 today (paper -> arxiv.org/abs/2406.03812)

152

Reposted by Tal Daniel

Aviv Tamar @aviv-tamar.bsky.social · 25/11/2024

Want to learn / teach RL?   Check out new book draft: Reinforcement Learning - Foundations sites.google.com/view/rlfound... W/ Shie Mannor & Yishay Mansour This is a rigorous first course in RL, based on our teaching at TAU CS and Technion ECE.

415435