iiinova — where your digital self keeps evolving

Juan Carlos Niebles @jcniebles.bsky.social · 10/09/2026

In case you missed it, here are my slides for my talk yesterday at CONTEXTUS @eccv.bsky.social www.niebles.net/blog/2026/ec... Thanks to the organizers for the invitation and the audience for the engaging questions!

000 0

Juan Carlos Niebles @jcniebles.bsky.social · 25/08/2026

I'm hiring! Looking for a Director of Vision Intelligence to join my team at @sumsung.bsky.social Samsung Research America 🚀 If you’re passionate about multimodal AI, computer vision, and bringing advanced research directly onto everyday devices, let’s connect! sra.samsung.com/careers/deta...

sra.samsung.com

Career Detail – Samsung Research America

000 0

Reposted by Juan Carlos Niebles

Salesforce AI Research @sfresearch.bsky.social · 02/06/2026

6th Multimodal Algorithmic Reasoning Workshop at #CVPR2026 is Thursday (6/4), 8:55 AM–12:30 PM MDT, Room 601, Colorado Convention Center. 👥 Keynote by Juan Carlos Niebles @jcniebles.bsky.social, organized by Honglu Zhou @hongluzhou.bsky.social 📆 sforce.co/4ueOT7j

011 0

Juan Carlos Niebles @jcniebles.bsky.social · 26/04/2026

Heading to the DATA-FM workshop at #ICLR2026? 🦾 Catch my invited talk at 1:30 PM today in Room 203A+B: "Agentic Ambient Intelligence: Perception, Reasoning & Action." I’ll be diving into 4 new research pillars for moving AI into the physical world. Come say hi! 📍

110 0

Juan Carlos Niebles @jcniebles.bsky.social · 26/04/2026

Nice to re connect in person, enjoy Brazil!

020 0

Juan Carlos Niebles @jcniebles.bsky.social · 04/12/2025

Our #NeurIPS2025 oral presentation is starting in a few minutes! Join us: ⏰3:30 pm 📍 Ballroom 6AB grafting.stanford.edu arxiv.org/abs/2506.05340

grafting.stanford.edu

Exploring Diffusion Transformer Designs via Grafting

000 0

Juan Carlos Niebles @jcniebles.bsky.social · 21/10/2025

Such a great two days of workshops! Fuelled by inspiring talks and excellent reconnections with friends and colleagues—definitely my favorite part of the conference. Missed my talk on AI Agents: from Language to Multimodal Reasoning? Summary and slides are here: www.niebles.net/blog/2025/mm...

niebles.net

AI Agents: From Language to Multimodal Reasoning | Juan Carlos Niebles

An outline of our recent journey on AI Agents

010 0

Juan Carlos Niebles @jcniebles.bsky.social · 21/10/2025

Talk is done! Shared our work on Multimodal AI Agents at the #ICCV2025 Workshop on Multi-Modal Reasoning. 🤖 All the slides, key papers, and the research journey are consolidated in this new blog post: 📄https://www.niebles.net/blog/2025/mmagents/ @iccv.bsky.social

020 0

Juan Carlos Niebles @jcniebles.bsky.social · 20/10/2025

We will be presenting Strefer today at Poster 52 9:30-10:30am, join us to learn more about our work on Video-Languange at @salesforce.com AI Research @iccv.bsky.social #ICCV2025 strefer.github.io arxiv.org/abs/2509.03501

strefer.github.io

Strefer: Data Engine for Video LLMs

041 0

Juan Carlos Niebles @jcniebles.bsky.social · 22/09/2025

📢📢 Exciting news! Our paper, "Exploring Diffusion Transformer Designs via Grafting," has been accepted as an Oral at #NeurIPS2025, with only 77 out of 21k submissions receiving this honor. 📄Paper: arxiv.org/abs/2506.05340 🌎Website: grafting.stanford.edu 🧑🏻‍💻Code: github.com/keshik6/graf...

grafting.stanford.edu

Exploring Diffusion Transformer Designs via Grafting

090 0

Juan Carlos Niebles @jcniebles.bsky.social · 04/09/2025

Strefer: our new work for auto-generating instruction data on space–time–focused video tasks: spatiotemporal reasoning, space-time reference understanding, etc. for Video LLMs ✅ Auto & scalable ✅ Fine-grained, space–time–grounded queries ✅ Effective 📄: arxiv.org/abs/2509.03501 🌐: strefer.github.io

arxiv.org

Strefer: Empowering Video LLMs with Space-Time Referring and Reasoning via Synthetic Instruction Data

Next-generation AI companions must go beyond general video understanding to resolve spatial and temporal references in dynamic, real-world environments. Existing Video Large Language Models (Video LLM...

131 0

Juan Carlos Niebles @jcniebles.bsky.social · 17/06/2025

Check out a new episode of The AI Research Lab - Explained on Multimodal AI. Had a blast creating this with the @salesforce.com team! youtu.be/r98jGdLtO6Q

youtu.be

What is Multimodal AI? | The AI Research Lab - Explained

YouTube video by Salesforce

030 0

Juan Carlos Niebles @jcniebles.bsky.social · 12/06/2025

Congrats Chaitanya on winning the BEST PAPER AWARD 🥇 🏆 Check out details of our work: arxiv.org/abs/2504.12513

030 0

Juan Carlos Niebles @jcniebles.bsky.social · 12/06/2025

Our first #cvpr2025 poster is up! 🕐Come check it out right now until 13:00 “AdaVid: Adaptive Video-Language Pretraining” 🪧ExHall D Poster # 203 📝 arxiv.org/abs/2504.12513

020 0

Juan Carlos Niebles @jcniebles.bsky.social · 11/06/2025

Just finished a day at the #CVPR2025 Area Chair workshop. Lots of interesting discussions and ideas, reconnection with colleagues and friends. Had the chance to present our ViUnit poster to fellow ACs. If you missed it, come to our Sunday poster session. See details in the 🧵⬇️

030 0

Juan Carlos Niebles @jcniebles.bsky.social · 10/06/2025

If you're at #CVPR2025, please stop by my posters and say hello! I'd love to chat about our work and all things computer vision. See you in Nashville! 👋

010 0

Juan Carlos Niebles @jcniebles.bsky.social · 10/06/2025

Next, "Re-thinking Temporal Search for Long-Form Video Understanding" #CVPR2025 🗓️ Fri Jun 13, 4PM-6PM 📍 ExHall D Poster #306 🔗 Paper: arxiv.org/abs/2504.02259 🌐 Website: longvideohaystack.github.io 💻 Code: github.com/LongVideoHay... 📊 Data: huggingface.co/datasets/LVH... #VideoUnderstanding

arxiv.org

Re-thinking Temporal Search for Long-Form Video Understanding

Efficiently understanding long-form videos remains a significant challenge in computer vision. In this work, we revisit temporal search paradigms for long-form video understanding and address a fundam...

111 0

Juan Carlos Niebles @jcniebles.bsky.social · 10/06/2025

I'll also be presenting multiple papers at #CVPR2025! First up: "AdaVid: Adaptive Video-Language Pretraining". 🗓️ Thu Jun 12, 12:00-13:00PM 📍 ExHall D Poster #202 🔗 Paper: arxiv.org/abs/2504.12513 🌐 Website: chaitanya100100.github.io/AdaVid/ #VideoLanguage #Pretraining

arxiv.org

AdaVid: Adaptive Video-Language Pretraining

Contrastive video-language pretraining has demonstrated great success in learning rich and robust video representations. However, deploying such video encoders on compute-constrained edge devices rema...

111 0

Juan Carlos Niebles @jcniebles.bsky.social · 10/06/2025

Kicking things off on June 11th by participating in the #CVPR2025 Area Chair workshop! Eager to connect with fellow ACs and colleagues. Let's make this an impactful conference!

100 0

Juan Carlos Niebles @jcniebles.bsky.social · 10/06/2025

Excited to attend #CVPR2025 in Nashville! 🤠 Looking forward to a fantastic week of cutting-edge computer vision research and connecting with the community. @cvprconference.bsky.social

130 0

Juan Carlos Niebles @jcniebles.bsky.social · 04/06/2025

This RL approach effectively aligns VLMs with the demands of interactive decision-making. It's a powerful new pathway for developing more capable and adaptable visual agents using readily available VLM tech.

100 0

Juan Carlos Niebles @jcniebles.bsky.social · 04/06/2025

We tested our approach on PaliGemma, xGen-MM, and MoonDream2 across Gym Cards, BabyAI, and MiniWoB. Results? Substantial improvements in valid action syntax accuracy and task success rates, even starting from noisy data!

100 0

Juan Carlos Niebles @jcniebles.bsky.social · 04/06/2025

This approach works great for offline-to-online fine-tuning, learning from static datasets (even random actions!) and then smoothly transitioning to online learning where the agent gathers new data to refine its policy. Self-improvement is key!

110 0

Juan Carlos Niebles @jcniebles.bsky.social · 04/06/2025

AFSFT helps VLMs overcome challenges like strict action syntax and suboptimal data. It learns from demonstrations and filters out tokens that would lead to invalid syntax or poor choices, even penalizing invalid syntax.

100 0

Juan Carlos Niebles @jcniebles.bsky.social · 04/06/2025

Enter Reinforcement Learning (RL)! Our paper introduces an "offline-to-online" RL technique called Advantage-Filtered Supervised Fine-Tuning (AFSFT) that allows VLMs to learn through trial and error, improving even with imperfect initial data.

110 0

Juan Carlos Niebles @jcniebles.bsky.social · 04/06/2025

Traditional supervised fine-tuning (SFT) has limits – it can't go beyond its training data, and imperfect datasets mean replicating flaws. What if we don't have perfect examples or a good initial VLM?

100 0

Juan Carlos Niebles @jcniebles.bsky.social · 04/06/2025

The catch? VLMs can struggle with the precise rules and structured outputs many agent tasks require, unlike LLMs which excel at function calling and specific syntax. Think describing a button vs. knowing the exact command to click it.

100 0

Juan Carlos Niebles @jcniebles.bsky.social · 04/06/2025

Large Language Models (LLMs) are great for agents, but what happens when we give them "eyes"? VLMs extend this power to process visual info, opening up new possibilities like robotic control and automating tasks by "seeing" your screen.

100 0

Juan Carlos Niebles @jcniebles.bsky.social · 04/06/2025

Just dropped a new blog post: "Level up your Agents: Teaching Vision-Language Models to Play by the Rules"! We're exploring how to make Vision-Language Models (VLMs) even smarter at interactive tasks. blog: www.niebles.net/blog/2025/vl... arxiv: arxiv.org/abs/2505.03181 #multimodalAI #agents #VLM

121 0

Juan Carlos Niebles @jcniebles.bsky.social · 12/05/2025

Check out this great intro to Large Action Models, the key engine powering the AI Agent revolution. 🤖 By @salesforce.com AI Research’s Shelby Heinecke. See video here: youtube.com/watch?v=vlvv...

youtube.com

What Are Large Action Models? | The AI Research Lab - Explained

YouTube video by Salesforce

010 0

Reposted by Juan Carlos Niebles

Kathy Baxter @baxterkb.bsky.social · 12/05/2025

@salesforce.com #AI Research has a new series called "AI Explained." 🎬 "The AI Research Lab - Explained" debuts with our groundbreaking work on Large Action Models! Sr. Mgr Shelby Heinecke reveals how we're training these specialized models to generate precise, executable actions. t.co/XLhlN2EZyk

t.co

https://bit.ly/4kfipp4

031 0

Reposted by Juan Carlos Niebles

#CVPR2026 @cvprconference.bsky.social · 10/05/2025

Behind every great conference is a team of dedicated reviewers. Congratulations to this year’s #CVPR2025 Outstanding Reviewers! cvpr.thecvf.com/Conferences/...

04712 0

Juan Carlos Niebles @jcniebles.bsky.social · 07/05/2025

Will AI be a "bicycle for the mind" boosting our creativity, or could it overshadow our own abilities? 🤔 📝 My latest blog explores this fascinating question! Read more here: www.niebles.net/blog/2025/cr... #AI #creativity #artificialintelligence

niebles

010 0

Juan Carlos Niebles @jcniebles.bsky.social · 24/04/2025

Unfortunately not my experience… bsky.app/profile/jcni...

110 0

Juan Carlos Niebles @jcniebles.bsky.social · 24/04/2025

You are lucky. I still need to chase reviewers and have had to assign 12 emergency reviews for my pile!

050 0

Juan Carlos Niebles @jcniebles.bsky.social · 22/04/2025

With AI models trained in colossal datasets, does the traditional concept of “generalization” (performing well on *unseen* data) still hold? My latest blog outlines this critical question. Join the discussion! #AI #MachineLearning #Generalization www.niebles.net/blog/2025/ga...

341 0

Reposted by Juan Carlos Niebles

hongluzhou.bsky.social @hongluzhou.bsky.social · 18/04/2025

Always love Juan Carlos's blogs! 😍 His latest dives into how our #CVPR2025 work, ViUniT, catches sneaky bugs in visual programs using synthetic visual unit tests. Fewer lucky mistakes, better and more reliable results, and even small open models beating GPT-4o-mini. Super cool stuff 👇

021 0

Juan Carlos Niebles @jcniebles.bsky.social · 18/04/2025

New blog post: "Are your Visual Programs Right for the Wrong Reasons?" 🤔 Dive into the motivation behind our @cvprconference.bsky.social #CVPR2025 paper! 📰 Blog: www.niebles.net/blog/2025/vi... ➡️ Project: artemisp.github.io/viunit/ 📄 Paper: arxiv.org/abs/2412.08859 Work by Artemis P & Honglu Z

010 0

Juan Carlos Niebles @jcniebles.bsky.social · 18/04/2025

New blog post: "Are your Visual Programs Right for the Wrong Reasons?" 🤔 Dive into the motivation behind our @cvprconference.bsky.social #CVPR2025 paper! 📰 Blog: niebles.net/blog/2025/vi... ➡️ Project: artemisp.github.io/viunit/ 📄 Paper: arxiv.org/abs/2412.08859 Work by Artemis P & Honglu Z

130 0

Juan Carlos Niebles @jcniebles.bsky.social · 03/04/2025

Congrats on getting it done, and best wishes for your sabbatical!

110 0

Juan Carlos Niebles @jcniebles.bsky.social · 27/03/2025

Just around the corner! 🥁 Stay tuned for the latest from the AI Index. ⬇️⬇️

010 0

Juan Carlos Niebles @jcniebles.bsky.social · 24/03/2025

Exciting news! The 2025 AI Index is coming soon! 🚀 Get ready for a data-driven look at the latest trends and progress in AI. If you want to understand the field, this is your go-to resource. Stay tuned for the release! #AI #ArtificialIntelligence #AIIndex2025

020 0

Reposted by Juan Carlos Niebles

#CVPR2026 @cvprconference.bsky.social · 28/02/2025

Only 1.6k followers? We can do BETTER! 😉

4136 0