iiinova — where your digital self keeps evolving

Soumith Chintala @soumithchintala.bsky.social · 17/03/2025

A few months ago we quietly open-sourced a PyTorch video decoding library called torchcodec -- small, nimble, fast, supports GPU decoding via ffmpeg. The Hugging Face folks had some nice things to say about it as they integrated it into LeRobot. Check it out here: github.com/pytorch/torc...

0454

Soumith Chintala @soumithchintala.bsky.social · 27/02/2025

the Aria 2 glasses are pretty great for robot data collection. they're also getting really good for general agentic use... Read the full announcement here: www.meta.com/blog/project...

1142

Soumith Chintala @soumithchintala.bsky.social · 01/01/2025

i added an example here: bsky.app/profile/soum...

150

Soumith Chintala @soumithchintala.bsky.social · 01/01/2025

what I'm finding is that, the models want to be more of an artist than a replacement for photoshop -- which is fine, but I want to be the artist here, and want the tool to be more of a "magically easier photoshop where I ask it what to do in detail, and it does that -- not more not less"

290

Soumith Chintala @soumithchintala.bsky.social · 01/01/2025

i'll give a representative but not exact example: change the color of X's shirt from blue to red: the generations often change the entire shirt style itself -- they don't respect how much and what I'm trying to change, and dont try to preserve details I ask to preserve

280

Soumith Chintala @soumithchintala.bsky.social · 01/01/2025

what are AI products that allow me to transform existing images, while preserving some selective details (that i select), like faces, areas, etc.? the tools I've used so far only take the selection as a hint, or dont generate well around the selection? trying personalized art

3170

Soumith Chintala @soumithchintala.bsky.social · 20/12/2024

3. They've also made it easy to load MJCF and other common specs used in robotics. They've also made visualization work out of the box (they hacked up a hybrid of pyrender, pyglet and LuisaRender with a ton of their own patches).

040

Soumith Chintala @soumithchintala.bsky.social · 20/12/2024

2. The APIs are reasonably simple and well-designed, and they did take out the cross-platform pain in many ways -- CPU, CUDA, Metal etc. are all supported across Linux, OSX, Windows -- thanks to Taichi (and to a small part PyTorch).

130

Soumith Chintala @soumithchintala.bsky.social · 20/12/2024

1. It's nice that the internals are written with Taichi, so all the sim code is written in python, more accessible and easy-to-read than retrofitting physics on top of a Tensor compiler (like mujoco did with MJX) and possibly faster because Taichi is a more suited DSL / compiler.

110

Soumith Chintala @soumithchintala.bsky.social · 20/12/2024

The whole GenAI/LLM/VLM stuff seems to be unreleased or "aspirational". My favorite aspects:

110

Soumith Chintala @soumithchintala.bsky.social · 20/12/2024

It's basically like Mujoco but with more advanced materials/rendering/solvers, written all in Python thanks to being powered by Taichi, which makes it much more accessible. I like it a lot. It's very accessible. They went too far with marketing, but willing to ignore it for now.

171

Soumith Chintala @soumithchintala.bsky.social · 20/12/2024

i rabbit-holed into the Genesis Sim codebase because it went viral on X, and the website is hypey and unclear; and I didn't want to just blindly retweet. genesis-embodied-ai.github.io 👇

genesis-embodied-ai.github.io

Genesis

1403

Soumith Chintala @soumithchintala.bsky.social · 20/12/2024

also, congrats OpenAI on O3, and thank you for rapidly making progress on intelligence.

011

Soumith Chintala @soumithchintala.bsky.social · 20/12/2024

Models are dumb as rock without the right context -- pretrained context doesn't help with day-to-day or specialized things. Private ecosystems and company bureaucracies means you have to feed the models your own context for the next X years....unless computer-use gets ready. Cant wait for it!

161

Soumith Chintala @soumithchintala.bsky.social · 20/12/2024

intelligence is starting to get good, but context is still siloed for stupid reasons. get models that do human-level computer-use already, please...!

1122

Soumith Chintala @soumithchintala.bsky.social · 18/12/2024

Glean for personal/self-hosted: is there an open source / self-hosted project that integrates pulling context from gmail, docs, sheets, calendar, whatsapp, ig, imessage, etc.?

1152

Reposted by Soumith Chintala

Chris Paxton @cpaxton.bsky.social · 03/12/2024

I'd like to introduce what I've been working at @hellorobot.bsky.social: Stretch AI, a set of open-source tools for language-guided autonomy, exploration, navigation, and learning from demonstration. Check it out: github.com/hello-robot/... Thread ->

613223

Soumith Chintala @soumithchintala.bsky.social · 19/11/2024

so much detail, it's incredible that you've gotten this deep....twice ☺️!!!

140

Reposted by Soumith Chintala

Chris Paxton @cpaxton.bsky.social · 09/11/2024

Very excited about this new project, DynaMem. It allows our robots to function in previously unseen environments, performing long-horizon manipulation tasks. Most importantly it *generalizes*, meaning you can try it out on a wide variety of homes and on different objects. (4x video)

2316

Reposted by Soumith Chintala

Maike Osborne @maosbot.bsky.social · 09/11/2024

New here? Interested in AI/ML? Check out these great starter packs! AI: go.bsky.app/SipA7it RL: go.bsky.app/3WPHcHg Women in AI: go.bsky.app/LaGDpqg NLP: go.bsky.app/SngwGeS AI and news: go.bsky.app/5sFqVNS You can also search all starter packs here: blueskydirectory.com/starter-pack...

66558212

Soumith Chintala @soumithchintala.bsky.social · 16/11/2024

what are good starter packs for: AI researchers, AI Systems people, GenAI hackers, LLM enthusiasts?

7442