Sign in

Carlo Sferrazza

@carlosferrazza.bsky.social
110 followers 60 following 33 posts

Postdoc at Berkeley AI Research. PhD from ETH Zurich. Robotics, Artificial Intelligence, Humanoids, Tactile Sensing. sferrazza.cc

PostsRepliesMedia
Carlo Sferrazza @carlosferrazza.bsky.social · 29/05/2025
A very fun project at @ucberkeleyofficial.bsky.social, led by amazing Younggyo Seo, with Haoran Geng, Michal Nauman, Zhaoheng Yin, and Pieter Abbeel! Page: younggyo.me/fast_td3/ Arxiv: arxiv.org/abs/2505.22642 Code: github.com/younggyoseo/...
younggyo.me
FastTD3: Simple, Fast, and Capable Reinforcement Learning for Humanoid Control
FastTD3: Simple, Fast, and Capable Reinforcement Learning for Humanoid Control.
010
Carlo Sferrazza @carlosferrazza.bsky.social · 29/05/2025
FastTD3 is open-source, and compatible with most sim-to-real robotics frameworks, e.g., MuJoCo Playground and Isaac Lab. All the advances in scaling off-policy RL are now readily available to the robotics community 🤖
100
Carlo Sferrazza @carlosferrazza.bsky.social · 29/05/2025
A very cool thing: FastTD3 achieves state-of-the-art performance on most HumanoidBench tasks, even superior to model-based algorithms. All it takes: 128 parallel environments and 1-3 hours of training 🤯
100
Carlo Sferrazza @carlosferrazza.bsky.social · 29/05/2025
Off-policy methods have pushed RL sample efficiency, but robotics still leans on parallel on-policy RL (PPO) for wall-time gains. FastTD3 gets the best of both worlds!
100
Carlo Sferrazza @carlosferrazza.bsky.social · 29/05/2025
We just released FastTD3: a simple, fast, off-policy RL algorithm to train humanoid policies that transfer seamlessly from simulation to the real world. younggyo.me/fast_td3
130
Carlo Sferrazza @carlosferrazza.bsky.social · 22/04/2025
Heading to @ieeeras.bsky.social RoboSoft today! I'll be giving a short Rising Star talk Thu at 2:30pm: "Towards Multi-sensory, Tactile-Enabled Generalist Robot Learning" Excited for my first in-person RoboSoft after the 2020 edition went virtual mid-pandemic. Reach out if you'd like to chat!
030
Carlo Sferrazza @carlosferrazza.bsky.social · 17/04/2025
And co-organizers @sukhijab.bsky.social, @amyxlu.bsky.social, Lenart Treven, Parnian Kassraie, Andrew Wagenmaker, Olivier Bachem, @kjamieson.bsky.social, @arkrause.bsky.social, Pieter Abbeel
000
Carlo Sferrazza @carlosferrazza.bsky.social · 17/04/2025
With amazing speakers Sergey Levine, Dorsa Sadigh, @djfoster.bsky.social, @ji-won-park.bsky.social, Ben Van Roy, Rishabh Agarwal, @alisongopnik.bsky.social, Masatoshi Uehara
100
Carlo Sferrazza @carlosferrazza.bsky.social · 17/04/2025
What is the place of exploration in today's AI landscape and in which settings can exploration algorithms address current open challenges? Join us to discuss this at our exciting workshop at @icmlconf.bsky.social 2025: EXAIT! exait-workshop.github.io #ICML2025
193
Carlo Sferrazza @carlosferrazza.bsky.social · 16/01/2025
Many robots (and robot videos!), and many awesome collaborators at @ucberkeleyofficial.bsky.social @uoft.bsky.social @cambridgeuni.bsky.social @stanforduniversity.bsky.social @deepmind.google.web.brid.gy – huge shoutout to the entire team!
020
Carlo Sferrazza @carlosferrazza.bsky.social · 16/01/2025
Very easy installation, it can even run on a single Python notebook: colab.research.google.com/github/googl... Check out @mujoco.bsky.social’s thread above for all the details. Can't wait to see the robotics community build on this pipeline and keep pushing the field forward!
colab.research.google.com
Google Colab
120
Carlo Sferrazza @carlosferrazza.bsky.social · 16/01/2025
It was really amazing to work on this and see the whole project come together. Sim-to-real is often an iterative process – Playground makes it seamless. An open-source ecosystem is essential for integrating new features – check out Madrona-MJX for distillation-free visual RL!
120
Carlo Sferrazza @carlosferrazza.bsky.social · 16/01/2025
Big news for open-source robot learning! We are very excited to announce MuJoCo Playground. The Playground is a reproducible sim-to-real pipeline that leverages MuJoCo ecosystem and GPU acceleration to learn robot locomotion and manipulation in minutes. playground.mujoco.org
120
Carlo Sferrazza @carlosferrazza.bsky.social · 13/01/2025
This work is the result of an amazing collaboration at @ucberkeleyofficial.bsky.social with the other co-leads Josh Jones and Oier Mees, as well as Kyle Stachowicz, Pieter Abbeel, and Sergey Levine! Paper: arxiv.org/abs/2501.04693 Website: fuse-model.github.io
arxiv.org
Beyond Sight: Finetuning Generalist Robot Policies with Heterogeneous Sensors via Language Grounding
Interacting with the world is a multi-sensory experience: achieving effective general-purpose interaction requires making use of all available modalities -- including vision, touch, and audio -- to fi...
000
Carlo Sferrazza @carlosferrazza.bsky.social · 13/01/2025
We open source the code and the models, as well as the dataset, which comprises 27k (!) action-labeled robot trajectories with visual, inertial, tactile, and auditory observations. Code: github.com/fuse-model/F... Models and dataset: huggingface.co/oier-mees/FuSe
github.com
GitHub - fuse-model/FuSe
Contribute to fuse-model/FuSe development by creating an account on GitHub.
100
Carlo Sferrazza @carlosferrazza.bsky.social · 13/01/2025
We find that the same general recipe is applicable to generalist policies with diverse architectures, including a large 3B VLA with a PaliGemma vision-language-model backbone.
100
Carlo Sferrazza @carlosferrazza.bsky.social · 13/01/2025
FuSe policies reason jointly over vision, touch, and sound, enabling tasks such as multimodal disambiguation, generation of object descriptions upon interaction, and compositional cross-modal prompting (e.g., “press the button with the same color as the soft object”).
100
Carlo Sferrazza @carlosferrazza.bsky.social · 13/01/2025
Pretrained generalist robot policies finetuned on multimodal data consistently outperform baselines finetuned only on vision data. This is particularly evident in tasks with partial visual observability, such as grabbing objects from a shopping bag.
100
Carlo Sferrazza @carlosferrazza.bsky.social · 13/01/2025
We use language instructions to ground all sensing modalities by introducing two auxiliary losses. In fact, we find that naively finetuning on a small-scale multimodal dataset results in the VLA over-relying on vision, ignoring much sparser tactile and auditory signals.
100
Carlo Sferrazza @carlosferrazza.bsky.social · 13/01/2025
Ever wondered what robots 🤖 could achieve if they could not just see – but also feel and hear? Introducing FuSe: a recipe for finetuning large vision-language-action (VLA) models with heterogeneous sensory data, such as vision, touch, sound, and more. Details in the thread 👇
111
Reposted by Carlo Sferrazza
Bhavya Sukhija @sukhijab.bsky.social · 17/12/2024
Excited to share MaxInfoRL, a family of powerful off-policy RL algorithms! The core focus of this work was to develop simple, flexible, and scalable methods for principled exploration. Check out the thread below to see how MaxInfoRL meets these criteria while also achieving SOTA empirical results.
151
Carlo Sferrazza @carlosferrazza.bsky.social · 17/12/2024
Work led by amazing @sukhijab.bsky.social at @ucberkeleyofficial.bsky.social AI Research, w/ Stelian Coros, @arkrause.bsky.social , and Pieter Abbeel! Paper: arxiv.org/abs/2412.12098 Website: sukhijab.github.io/projects/max...
arxiv.org
MaxInfoRL: Boosting exploration in reinforcement learning through information gain maximization
Reinforcement learning (RL) algorithms aim to balance exploiting the current best strategy with exploring new options that could lead to higher rewards. Most common RL algorithms use undirected explor...
000
Carlo Sferrazza @carlosferrazza.bsky.social · 17/12/2024
We are also excited to share both Jax and Pytorch implementations, making it simple for RL researchers to integrate MaxInfoRL into their training pipelines. Jax (built on jaxrl): github.com/sukhijab/max... Pytorch (based on @araffin.bsky.social‘s SB3): github.com/sukhijab/max...
github.com
GitHub - sukhijab/maxinforl_jax
Contribute to sukhijab/maxinforl_jax development by creating an account on GitHub.
110
Carlo Sferrazza @carlosferrazza.bsky.social · 17/12/2024
By combining MaxInfoRL with DrQv2 and DrM, this achieves state-of-the-art model-free performance on hard visual control tasks such as DMControl humanoid and dog tasks, improving both sample efficiency and steady-state performance.
100
Carlo Sferrazza @carlosferrazza.bsky.social · 17/12/2024
MaxInfoRL is a simple, flexible, and scalable add-on to most RL advancements. We combine it with various algorithms, such as SAC, REDQ, DrQv2, DrM, and more – consistently showing improved performance over the respective backbones.
100
Carlo Sferrazza @carlosferrazza.bsky.social · 17/12/2024
While standard Boltzmann exploration (e.g., SAC) focuses only on action entropy, MaxInfoRL maximizes entropy in both state and action spaces! This proves to be crucial when dealing with complex exploration settings.
110
Carlo Sferrazza @carlosferrazza.bsky.social · 17/12/2024
The core principle is to balance extrinsic rewards with intrinsic exploration. MaxInfoRL achieves this by 1) using an ensemble of dynamics models to estimate information gain, and 2) incorporating this as an automatically-tuned exploration bonus in addition to policy entropy.
100
Carlo Sferrazza @carlosferrazza.bsky.social · 17/12/2024
🚨 New reinforcement learning algorithms 🚨 Excited to announce MaxInfoRL, a class of model-free RL algorithms that solves complex continuous control tasks (including vision-based!) by steering exploration towards informative transitions. Details in the thread 👇
1182
Carlo Sferrazza @carlosferrazza.bsky.social · 17/12/2024
Work led by amazing @sukhijab.bsky.social at @ucberkeleyofficial.bsky.social AI Research, w/ Stelian Coros, @arkrause.bsky.social, and Pieter Abbeel! Paper: arxiv.org/abs/2412.12098 Website: sukhijab.github.io/projects/max...
arxiv.org
MaxInfoRL: Boosting exploration in reinforcement learning through information gain maximization
Reinforcement learning (RL) algorithms aim to balance exploiting the current best strategy with exploring new options that could lead to higher rewards. Most common RL algorithms use undirected explor...
000
Carlo Sferrazza @carlosferrazza.bsky.social · 17/12/2024
We are also excited to share both Jax and Pytorch implementations, making it simple for RL researchers to integrate MaxInfoRL into their training pipelines. Jax (built on jaxrl): github.com/sukhijab/max... Pytorch (based on @araffin.bsky.social‘s SB3): github.com/sukhijab/max...
github.com
GitHub - sukhijab/maxinforl_jax
Contribute to sukhijab/maxinforl_jax development by creating an account on GitHub.
100
Carlo Sferrazza @carlosferrazza.bsky.social · 17/12/2024
By combining MaxInfoRL with DrQv2 and DrM, this achieves state-of-the-art model-free performance on hard visual control tasks such as DMControl humanoid and dog tasks, improving both sample efficiency and steady-state performance.
100
Carlo Sferrazza @carlosferrazza.bsky.social · 17/12/2024
MaxInfoRL is a simple, flexible, and scalable add-on to most RL advancements. We combine it with various algorithms, such as SAC, REDQ, DrQv2, DrM, and more – consistently showing improved performance over the respective backbones.
100
Carlo Sferrazza @carlosferrazza.bsky.social · 17/12/2024
While standard Boltzmann exploration (e.g., SAC) focuses only on action entropy, MaxInfoRL maximizes entropy in both state and action spaces! This proves to be crucial when dealing with complex exploration settings.
100
Carlo Sferrazza @carlosferrazza.bsky.social · 17/12/2024
The core principle is to balance extrinsic rewards with intrinsic exploration. MaxInfoRL achieves this by 1) using an ensemble of dynamics models to estimate information gain, and 2) incorporating this as an automatically-tuned exploration bonus in addition to policy entropy.
100