iiinova — where your digital self keeps evolving

This project was part of my internship at Meta, and it was a great collaboration with Jia-Bin, Zhengqin, Zhao, Christian, Tuotuo, Michael, Johannes, Shenlong, and Changil 🙌

000

📍 Meet us at #CVPR2025 🗓️ June 13 (Fri.), 10:30–12:30 🪧 Come by our poster session to chat about IRIS and any research ideas Looking forward to reconnecting — and meeting new friends — in Nashville! 🎸✨

100

Please check out our paper, code, and more demo videos! 🌐 Project page: irisldr.github.io 💻 GitHub: github.com/facebookrese... 📝 Paper: arxiv.org/abs/2401.12977

100

Excited to share our work at #CVPR2025! 👁️IRIS estimates accurate surface material, spatially-varying HDR lighting, and camera response function given a set of LDR images! It enables realistic, view-consistent, and controllable relighting and object insertion. (links in 🧵)

Anand Bhattad @anandbhattad.bsky.social · 02/06/2025

Reposted by Chih-Hao Lin

I’m thrilled to share that I will be joining Johns Hopkins University’s Department of Computer Science (@jhucompsci.bsky.social, @hopkinsdsai.bsky.social) as an Assistant Professor this fall.

182

Reposted by Chih-Hao Lin

International Conference on 3D Vision @3dvconf.bsky.social · 29/05/2025

📢 3DV 2026 – Call for Papers is Out! 📝 Paper Deadline: Aug 18 🎥 Supplementary: Aug 21 🔗 3dvconf.github.io/2026/call-fo... 📅 Conference Date: Mar 20–23, 2026 🌆 Location: Vancouver 🇨🇦 🚀 Showcase your latest research to the world! #3DV2026 #CallForPapers #Vancouver #Canada

094

Reposted by Chih-Hao Lin

Jonathan Lorraine @jonlorraine.bsky.social · 09/05/2025

🔊 New NVIDIA paper: Audio-SDS 🔊 We repurpose Score Distillation Sampling (SDS) for audio, turning any pretrained audio diffusion model into a tool for diverse tasks, including source separation, impact synthesis & more. 🎧 Demos, audio examples, paper: research.nvidia.com/labs/toronto... 🧵below

161

Controllable Weather Simulation and Removal with Video Diffusion Models

This work is a great collaboration at NVIDIAAI by Chih-Hao Lin, Zian Wang, Ruofan Liang, Yuxuan Zhang, Sanja Fidler, Shenlong Wang, Zan Gojcic 🌐Please check out our project page: research.nvidia.com/labs/toronto...

research.nvidia.com

WeatherWeaver

010

The weather removal model successfully removes both transient (e.g., rain, snowflake) and persistent effects (e.g., puddle, snow cover), and can even restore sunny-day lighting from rainy/snowy videos.

WeatherWeaver combines two video diffusion models. The weather synthesis model generates realistic, temporally consistent weather, adapting shading naturally while preserving the original scene structure.

By combining and adjusting multiple weather effects, WeatherWeaver can simulate complex weather transitions, e.g. on 🌧️ rainy and ☃️ snowy days — without costly real-world acquisitions.

WeatherWeaver enables precise control of the weather effects by changing the intensity of the corresponding effects. 🌤️➡️🌥️

We train a video diffusion model to edit weather effects with precise control, using a novel data strategy combining synthetic videos, generative image editing, and auto-labeled real-world videos.

Realistic, controllable weather simulation opens new possibilities in 🚗 autonomous driving simulation and 🎬 filmmaking. Physics-based simulation requires accurate geometry and doesn’t scale to in-the-wild videos, while existing video editing often lacks realism and control.

Anand Bhattad @anandbhattad.bsky.social · 29/03/2025

What if you could control the weather in any video — just like applying a filter? Meet WeatherWeaver, a video model for controllable synthesis and removal of diverse weather effects — such as 🌧️ rain, ☃️ snow, 🌁 fog, and ☁️ clouds — for any input video.

131

Reposted by Chih-Hao Lin

[1/10] Is scene understanding solved? Models today can label pixels and detect objects with high accuracy. But does that mean they truly understand scenes? Super excited to share our new paper and a new task in computer vision: Visual Jenga! 📄 arxiv.org/abs/2503.21770 🔗 visualjenga.github.io

75914

Chih-Hao Lin @chih-hao.bsky.social · 22/03/2025

Check out our cool demos. The code is also open-source! Project website: haoyuhsu.github.io/autovfx-webs... Code (GitHub): github.com/haoyuhsu/aut... Paper: arxiv.org/abs/2411.02394

000

Chih-Hao Lin @chih-hao.bsky.social · 22/03/2025

🎬Imagine creating professional visual effects (VFX) with just words! We are excited to introduce AutoVFX, a framework that creates realistic video effects from natural language instructions! This is a cool project led by Hao-Yu, and we will present it at #3DV 2025!

Anand Bhattad @anandbhattad.bsky.social · 16/03/2025

Reposted by Chih-Hao Lin

Can we create realistic renderings of urban scenes from a single video while enabling controllable editing: relighting, object compositing, and nighttime simulation? Check out our #3DV2025 UrbanIR paper, led by @chih-hao.bsky.social that does exactly this. 🔗: urbaninverserendering.github.io

021

Reposted by Chih-Hao Lin

Jia-Bin Huang @jbhuang0604.bsky.social · 15/03/2025

Check out UrbanIR - Inverse rendering of unbounded scenes from a single video! It’s a super cool project led by the amazing Chih-Hao! @chih-hao.bsky.social is a rising star in 3DV! Follow him! Learn more here👇

0102

🙏 Huge thanks to our amazing collaborators from UIUC & UMD: Bohan, Yi-Ting, Kuan-Sheng, David, Jia-Bin (@jbhuang0604.bsky.social) , Anand (@anandbhattad.bsky.social) , and Shenlong. This work wouldn't be possible without you all.

020

📢 Meet us at 3DV 2025 in Singapore! We’re excited to present UrbanIR at 3DV 2025 @3dvconf.bsky.social , come chat with us to discuss future directions!

130

Check out our interactive demos, and the code is open-source! Project website: urbaninverserendering.github.io Code (GitHub): github.com/chih-hao-lin... Paper: arxiv.org/abs/2306.09349

UrbanIR precisely controls the lighting, simulating different times of day without time-consuming tripod captures.

With estimated scene properties, UrbanIR integrates a physically-based shading model into neural field, rendering realistic videos from novel viewpoints and lighting conditions.

🔍 How does it work? UrbanIR reconstructs scene properties—geometry, albedo, and shading—through inverse rendering. Since this is a highly ill-posed problem, we leverage 2D priors to guide the optimization.