iiinova — where your digital self keeps evolving

Reposted by Lerrel Pinto

Robot Talk @robottalkpod.bsky.social · 21/05/2025

How is AI helping robots to generalise their skills to unfamiliar environments? 🤖 🏠 In the latest episode, I chatted to Prof. Lerrel Pinto (@lerrelpinto.com) from New York University about #robot learning and decision making. Available wherever you get your podcasts: linktr.ee/robottalkpod

021

Lerrel Pinto @lerrelpinto.com · 18/04/2025

We just released RUKA, a $1300 humanoid hand that is 3D-printable, strong, precise, and fully open sourced! The key technical breakthrough here is that we can control joints and fingertips of the robot **without joint encoders**. All we need here is self-supervised data collection and learning.

1287

Lerrel Pinto @lerrelpinto.com · 28/03/2025

When life gives you lemons, you pick them up. (trained with robotutilitymodels.com)

1154

Reposted by Lerrel Pinto

Robot Talk @robottalkpod.bsky.social · 18/03/2025

What would you love to know about #robot learning and decision making? Later this season, I'll be chatting to Prof. Lerrel Pinto (@lerrelpinto.com) from NYU about using machine learning to train robots to adapt to new environments. Send me your questions for Lerrel: robottalk.org/ask-a-question/

0137

Lerrel Pinto @lerrelpinto.com · 02/03/2025

Is there a word for the feeling when you want to cheer for the other team?

150

Lerrel Pinto @lerrelpinto.com · 28/02/2025

The robot behaviors shown below are trained without any teleop, sim2real, genai, or motion planning. Simply show the robot a few examples of doing the task yourself, and our new method, called Point Policy, spits out a robot-compatible policy!

1205

Reposted by Lerrel Pinto

Chris Paxton @cpaxton.bsky.social · 26/02/2025

This is important because the humble iPhone is one of the best accessories for embodied AI out there, if not actually the best. It's got a depth sensor, good camera, built-in internet, decent compute, and -- uniquely -- it has really good slam already built in.

3154

Lerrel Pinto @lerrelpinto.com · 26/02/2025

We just released AnySense, an iPhone app for effortless data acquisition and streaming for robotics. We leverage Apple’s development frameworks to record and stream: 1. RGBD + Pose data 2. Audio from the mic or custom contact microphones 3. Seamless Bluetooth integration for external sensors

23510

Reposted by Lerrel Pinto

Eugene Vinitsky 🍒 @eugenevinitsky.bsky.social · 23/02/2025

A useful “productivity” trick is to remind yourself that research should be fun and inspiring and if it’s not that something should change.

2727

Lerrel Pinto @lerrelpinto.com · 20/02/2025

Just found a new winner for the most hype-baiting, unscientific plot I have seen. (From the recent Figure AI release)

1376

Reposted by Lerrel Pinto

Eugene Vinitsky 🍒 @eugenevinitsky.bsky.social · 20/02/2025

One reason to be intolerant of misleading hype in tech and science is that tolerating the small lies and deception is how you get tolerance of big lies

418427

Lerrel Pinto @lerrelpinto.com · 18/02/2025

Thank you to @sloanfoundation.bsky.social for this generous award to our lab. Hopefully this will bring us closer to building truly general-purpose robots!

3234

Lerrel Pinto @lerrelpinto.com · 13/02/2025

A fun, clever idea from @upiter.bsky.social : treat code generation as a sequential editing problem -- this gives you loads of training data from synthetically editing existing code And it works! Higher performance on HumanEval, MBPP, and CodeContests across small LMs like Gemma-2, Phi-3, Llama 3.1

150

Lerrel Pinto @lerrelpinto.com · 31/01/2025

We have been working a bunch on offline world models. Pre-trained features from DINOv2 seem really powerful for modeling. I hope this opens up a whole set of applications for decision making and robotics! Check out the thread from @gaoyuezhou.bsky.social for more details.

040

Reposted by Lerrel Pinto

Eugene Vinitsky 🍒 @eugenevinitsky.bsky.social · 09/01/2025

If you’re in grad school, finding a therapist can be really helpful. The thing you’re doing is hard and it’s harder if you don’t have help managing imposter syndrome, stress, self esteem, and a whole bunch of other things.

56513

Reposted by Lerrel Pinto

Laine Nooney @ 🔥🗑️🔥 @lainenooney.bsky.social · 30/12/2024

omg a student somehow accidentally wrote an email addressed to a faculty-wide NYU listserv and my inbox is now a master class on who understands the difference between a listserv and an email chain

2035382932

Reposted by Lerrel Pinto

Florian Ederer @florianederer.bsky.social · 25/12/2024

Humans vs Ants: Problem-solving Skills

911632

Lerrel Pinto @lerrelpinto.com · 18/12/2024

At NYU Abu Dhabi today and in love how cat friendly the campus is!

2111

Reposted by Lerrel Pinto

Daniel Rolph Author @danielrolphauthor.bsky.social · 15/12/2024

This holiday season, take a moment to visit your local bookstore. It’s about more than finding a great book—it’s about supporting the small businesses that keep our communities thriving.

572367312

Reposted by Lerrel Pinto

Remi Cadene @remicadene.bsky.social · 15/12/2024

HOT 🔥 fastest, most precise, and most capable hand control setup ever... Less than $450 and fully open-source 🤯 by @huggingface, @therobotstudio, @NepYope This tendon-driven technology will disrupt robotics! Retweet to accelerate its democratization 🚀 A thread 🧵

37327

Reposted by Lerrel Pinto

Alfredo Canziani @alfcnz.bsky.social · 13/12/2024

Outstanding presentation, finally! DynaMo: In-Domain Dynamics Pretraining for Visuo-Motor Control @jeffacce.bsky.social @lerrelpinto.com

0112

Lerrel Pinto @lerrelpinto.com · 10/12/2024

New paper! We show that by using keypoint-based image representation, robot policies become robust to different object types and background changes. We call this method Prescriptive Point Priors for robot Policies or P3-PO in short. Full project is here: point-priors.github.io

1377

Lerrel Pinto @lerrelpinto.com · 09/12/2024

Modern policy architectures are unnecessarily complex. In our #NeurIPS2024 project called BAKU, we focus on what really matters for good policy learning. BAKU is modular, language-conditioned, compatible with multiple sensor streams & action multi-modality, and importantly fully open-source!

1309

Reposted by Lerrel Pinto

Raunaq Bhirangi @raunaqb.bsky.social · 09/12/2024

Robot utility models are not just among the first learned models that work zero-shot on a mobile manipulator, but also provide a nuanced discussion on what works and what doesn't in data-driven robot learning.

071

Lerrel Pinto @lerrelpinto.com · 08/12/2024

Since we are nearing the end of the year, I'll revisit some of our work I'm most excited about from the last year and maybe a sneak peek of what we are up to next. To start of, Robot Utility Models, which enables zero-shot deployment. In the video below, the robot hasnt seen these doors before.

2368

Reposted by Lerrel Pinto

Chris Paxton @cpaxton.bsky.social · 03/12/2024

I'd like to introduce what I've been working at @hellorobot.bsky.social: Stretch AI, a set of open-source tools for language-guided autonomy, exploration, navigation, and learning from demonstration. Check it out: github.com/hello-robot/... Thread ->

613223

Reposted by Lerrel Pinto

Jia-Bin Huang @jbhuang0604.bsky.social · 01/12/2024

How to drive your research forward? “I tested the idea we discussed last time. Here are some results. It does not work. (… awkward silence)” Such conversations happen so many times when meetings with students. How do we move forward? You need …

19118

Reposted by Lerrel Pinto

Grace Lindsay @neurograce.bsky.social · 27/11/2024

Then "ten simple rules" series from plos might count? It is a little bit more about how science is done, but insofar as it aims to write down practical tips that are often just spread by word of mouth it could be a good inspiration: collections.plos.org/collection/t...

collections.plos.org

Ten Simple Rules - PLOS Collections

"Ten Simple Rules" provide a quick, concentrated guide for mastering some of the professional challenges research scientists face in their careers.

031

Lerrel Pinto @lerrelpinto.com · 27/11/2024

I think we need an AMA series for Robotics / Embodied AI with an optional anonymous setting. Will be both fun and informative to new community members to absorb folk knowledge.

5425

Reposted by Lerrel Pinto

Cathy Wu @cathywu.bsky.social · 27/11/2024

I collected some folk knowledge for RL and stuck them in my lecture slides a couple weeks back: web.mit.edu/6.7920/www/l... See Appendix B... sorry, I know, appendix of a lecture slide deck is not the best for discovery. Suggestions very welcome.

web.mit.edu

311418

Reposted by Lerrel Pinto

Ken Goldberg @ken-goldberg.bsky.social · 26/11/2024

Interesting article but the author drank the Kool-Aid and never sought out other viewpoints: “Foundation models like GPT-4 have largely subsumed [previous] models that help robots with planning and vision, and locomotion and dexterity will probably soon be subsumed, too.”

1274

Lerrel Pinto @lerrelpinto.com · 26/11/2024

Nice work Remi and team. We need more of this!

081

Reposted by Lerrel Pinto

Aviv Tamar @aviv-tamar.bsky.social · 25/11/2024

Want to learn / teach RL?   Check out new book draft: Reinforcement Learning - Foundations sites.google.com/view/rlfound... W/ Shie Mannor & Yishay Mansour This is a rigorous first course in RL, based on our teaching at TAU CS and Technion ECE.

415435

Lerrel Pinto @lerrelpinto.com · 25/11/2024

Wonderful blog post by Matt Mason: mtmason.com/the-beetle-r... Quote from it: “... When humans have to handle things not designed for their hands, it’s all tools! Look at their dentists and surgeons. Look at their watchmakers. ...”

mtmason.com

The Beetle Roboticists: A Parable - Matthew T. Mason

How do we assess ideas about humanoids, without our anthropocentric bias? Maybe if the ideas were being pushed by another species. Beetles?

1218

Lerrel Pinto @lerrelpinto.com · 24/11/2024

We got stranded for a day in rural NY without electricity, running water, heat, internet, or cell service. It is crazy how difficult it is to live without these relatively modern inventions. I hope one day robots will join this list.

1233

Reposted by Lerrel Pinto

Thomas Wood @phytomech.com · 18/11/2024

Super impressive. Can't wait to see how you're doing teleoperation. This is the best option I've found that uses VR (by @lerrel.bsky.social's team). open-teach.github.io

open-teach.github.io

OPEN TEACH: A Versatile Teleoperation System for Robotic Manipulation

171

Reposted by Lerrel Pinto

Jo Wolfe, PhD @jopabinia.bsky.social · 01/08/2024

Our paper on convergent evolution of land crabs is out in its final final form. The journal didn't pick my image as the cover, so I'm sharing it here because it must be seen! academic.oup.com/sysbio/artic... 🦀🦑🧪

Circular timetree of crabs with pie slices colored by superfamily (Figure 2 in the paper). Simplified version with no text/labels. Line drawings of crabs arranged around the perimeter like a zodiac, with colors matching the superfamily pie slice.
Figure by me, drawings by Harrison Mancke and Javi Luque

841509365

Reposted by Lerrel Pinto

Eugene Vinitsky 🍒 @eugenevinitsky.bsky.social · 13/11/2024

Postdoctoral fellowships are open at NYU. Reach out if you're interested in having me as a mentor: apply.interfolio.com/158723 Some non-exclusive topics of interest: - Scaling up RL - Multi-agent planning - Equilibrium computation

apply.interfolio.com

Apply - Interfolio {{$ctrl.$state.data.pageTitle}} - Apply - Interfolio

2234

Reposted by Lerrel Pinto

Chris Paxton @cpaxton.bsky.social · 14/11/2024

In Robot Utility Models, we collected a large number of demonstration videos — 1000 examples in 40 environments — using a tool called “the Stick." The lead author even demonstrated it live at CoRL on completely new furniture. This one really does 100% actually work - I’ve used it in my house.

1195

Lerrel Pinto @lerrelpinto.com · 14/11/2024

Hi, I am new here and would love to interact with folks interested in robotics & AI. A bit about myself: I currently run a robotics lab at NYU, where we are building general-purpose robots. Some of our latest projects are on www.lerrelpinto.com. If you have questions ask away!

6578

Reposted by Lerrel Pinto

Chris Paxton @cpaxton.bsky.social · 09/11/2024

Very excited about this new project, DynaMem. It allows our robots to function in previously unseen environments, performing long-horizon manipulation tasks. Most importantly it *generalizes*, meaning you can try it out on a wide variety of homes and on different objects. (4x video)

2316

Lerrel Pinto @lerrelpinto.com · 01/11/2024

In our latest project, we train robots to mimic a human video of the task by matching the object features using RL. We only need one video and under an hour of robot training. Project was led by Irmak Guzey w/ Yinlong Dai, Georgy Savva and Raunaq Bhirangi. More details: object-rewards.github.io

061

Lerrel Pinto @lerrelpinto.com · 25/10/2024

It is really hard to get robot policies that are both precise (small margins for error) and general (robust to variations). We just released ViSk, where skin sensing is used to train fine-grained policies with ~1 hour of data. I have attached a single-take video on this post. visuoskin.github.io

2101

Reposted by Lerrel Pinto

Grace Lindsay @neurograce.bsky.social · 10/10/2024

New paper out! 🤖 I had the good luck of advising Nikhil Bhattasali on this work modeling spinal circuits in quadrupeds and showing how these bio-inspired architectures confer efficiency and generalization benefits in robots! arxiv.org/abs/2410.07174 #NeuroAI #compneurosky #neuroskyence #MLsky

35718