iiinova — where your digital self keeps evolving

Matthias Niessner @niessner.bsky.social · 23/12/2025

significantly outperforms generic visual pretraining (e.g., DINO-style features) in terms of generalization. 🌍https://simongiebenhain.github.io/Pix2NPHM 🎥https://youtu.be/MgpEJC5p1Ts Great work by Simon Giebenhain, Tobias Kirschstein, Liam Schoneveld, Davide Davoli, Zhe Chen.

000

Matthias Niessner @niessner.bsky.social · 23/12/2025

(1) large-scale registration of existing 3D head datasets, and (2) self-supervised training on vast in-the-wild 2D video datasets using pseudo ground-truth surface normals. Finally, we show that geometry-aware pretraining on pixel-aligned reconstruction tasks

100

Matthias Niessner @niessner.bsky.social · 23/12/2025

Pix2NPHM obtains fast and reliable NPHM reconstructions on real-world data. Inference-time optimization against surface normals and canonical point maps can further increase fidelity. Key to successful and generalized training of our ViT-based network are:

100

Matthias Niessner @niessner.bsky.social · 23/12/2025

Face tracking & 3D reconstruction are often limited by the representational capacity of PCA-based face models. By lifting NPHMs to a first-class reconstruction primitive, we enable more accurate geometry, richer expressions, and finer animation control.

100

Matthias Niessner @niessner.bsky.social · 17/12/2025

🌍https://peter-kocsis.github.io/IntrinsicImageFusion 🎥https://youtu.be/-Vs3tR1Xl7k Great work by Peter Kocsis and Lukas Hollein!

000

Matthias Niessner @niessner.bsky.social · 17/12/2025

3) optimize low-dimensional parameters for physically-grounded reconstructions. The results are relightable PBR textures for 3D scenes: check out the result on a real-world 3D scan from the ScanNet++ dataset!

110

Matthias Niessner @niessner.bsky.social · 17/12/2025

📢 Intrinsic Image Fusion for Multi-View 3D Material Reconstruction 📢 We combine generative material priors with inverse path tracing: 1) define a parametric texture space 2) fuse monocular predictions across views into consistent textures

191

Matthias Niessner @niessner.bsky.social · 16/12/2025

Today in our TUM AI - Lecture Series we'll have the amazing Ruiqi Gao, Google DeepMind. She'll talk about "𝐁𝐮𝐢𝐥𝐝𝐢𝐧𝐠 𝐠𝐞𝐧𝐞𝐫𝐚𝐭𝐢𝐯𝐞 𝐰𝐨𝐫𝐥𝐝 𝐦𝐨𝐝𝐞𝐥𝐬: progress and challenges". Live stream: www.youtube.com/live/CkOSMqw... 7pm GMT+1 / 10am PST (Tue Dec 16th).

youtube.com

TUM AI Lecture Series - Building generative world models: progress and challenges (Ruiqi Gaoi)

Abstract: Equipping AI models with the ability to imagine, reason, and act in the physical world is a crucial step toward achieving Artificial General Intelligence (AGI). Generative world models, whic...

082

Matthias Niessner @niessner.bsky.social · 05/11/2025

We also provide an interactive GUI to enable the exploration of our editing pipeline. 🌍 antoniooroz.github.io/PercHead/ 📽️ youtu.be/4hFybgTk4kE Great work by Antonio Oroz and and Tobias Kirschstein

antoniooroz.github.io

PercHead: Perceptual Head Model for Single-Image 3D Head Reconstruction & Editing

010

Matthias Niessner @niessner.bsky.social · 05/11/2025

by swapping the encoder, we can transform the model into a disentangled 3D editing pipeline. In this scenario, we can control geometry through - potentially hand-drawn - segmentation maps, and condition style via image or text prompt.

110

Matthias Niessner @niessner.bsky.social · 05/11/2025

Our trained reconstruction model is able to generate 3D-consistent heads from a single input image. Even with challenging side-view inputs, the model robustly infers missing regions for a coherent, high-fidelity output. In addition, our architecture seamlessly adapts to downstream tasks:

youtu.be

PercHead: Perceptual Head Model for Single-Image 3D Head Reconstruction & Editing

YouTube video by Matthias Niessner

100

Matthias Niessner @niessner.bsky.social · 05/11/2025

At its core is a generalized 3D head decoder trained with perceptual supervision from DINOv2 and SAM 2.1. We find that our new perceptual loss formulation improves reconstruction fidelity compared to commonly-used methods such as LPIPS.

antoniooroz.github.io

PercHead: Perceptual Head Model for Single-Image 3D Head Reconstruction & Editing

100

Matthias Niessner @niessner.bsky.social · 05/11/2025

📢📢 𝐏𝐞𝐫𝐜𝐇𝐞𝐚𝐝: 𝐏𝐞𝐫𝐜𝐞𝐩𝐭𝐮𝐚𝐥 𝐇𝐞𝐚𝐝 𝐌𝐨𝐝𝐞𝐥 𝐟𝐨𝐫 𝐒𝐢𝐧𝐠𝐥𝐞-𝐈𝐦𝐚𝐠𝐞 𝟑𝐃 𝐇𝐞𝐚𝐝 𝐑𝐞𝐜𝐨𝐧𝐬𝐭𝐫𝐮𝐜𝐭𝐢𝐨𝐧 & 𝐄𝐝𝐢𝐭𝐢𝐧𝐠📢📢 PercHead reconstructs realistic 3D heads from a single image and enables disentangled 3D editing via geometric controls and style inputs from images or text.

140

Matthias Niessner @niessner.bsky.social · 29/10/2025

#ICCV last week was incredible — catching up with so many people, chatting about research, and, most importantly, having lots of fun. Still hard to fathom this privilege as a researcher — getting to travel to such amazing places and be part of this brilliant community - Thanks!

030

Matthias Niessner @niessner.bsky.social · 26/10/2025

The hot topic at #ICCV2025 was World Models. They come in different flavors — (interactive) video models, neural simulators, reconstruction models, etc. — but the overarching goal is clear: Generative AI that predict and simulate how the real world works.

0150

Matthias Niessner @niessner.bsky.social · 24/10/2025

Hawaii on the same scale as the United Kingdom.

030

Matthias Niessner @niessner.bsky.social · 16/10/2025

𝐺𝑒𝑛𝑒𝑟𝑎𝑡𝑜 𝑒𝑟𝑔𝑜 𝑠𝑢𝑚 — I generate, therefore I am.

010

Matthias Niessner @niessner.bsky.social · 12/10/2025

On the bright side, tooling for training has dramatically improved since then. Deep learning frameworks (PyTorch et. al) and scheduling systems such as SLURM or Kubernetes have become the backbone of modern AI.

110

Matthias Niessner @niessner.bsky.social · 12/10/2025

Given the humongous compute demands of recent generative frontier AI models -- LLMs, image, and video models, etc. --, where compute is measured in Gigawatts, these challenges seem quite amusing.

100

Matthias Niessner @niessner.bsky.social · 12/10/2025

The required compute was typically a couple of GPUs on a single desktop machine, trained over several days; e.g., AlexNet was trained on two GTX 580 3GB GPUs for 5-6 days.

100

Matthias Niessner @niessner.bsky.social · 12/10/2025

In the 'early days' of modern deep learning (2012-2015) when ConvNets such as AlexNet or VGG came out, it was considered almost impractical to train an ImageNet classifier from scratch.

210

Matthias Niessner @niessner.bsky.social · 28/09/2025

Fantastic retreat this weekend by our research groups! Internal reviews, ideas brainstorming, paper reading, and much more! Of course also many social activities -- the highlight being our kayaking trip - lots of fun :)

0110

Matthias Niessner @niessner.bsky.social · 18/09/2025

All six of our submissions were accepted to #NeurIPS2025 🎉🥳 Awesome works about Gaussian Splatting Primitives, Lighting Estimation, Texturing, and much more GenAI :) Great work by Peter Kocsis, Yujin Chen, Zhening Huang, Jiapeng Tang, Nicolas von Lützow, Jonathan Schmidt 🔥🔥🔥

1100

Matthias Niessner @niessner.bsky.social · 17/09/2025

We generate multiple videos along short, pre-defined trajectories that explore the scene in depth. Our scene memory conditions each video on the most relevant prior views while avoiding collisions. Great work by Manuel Schneider & @LukasHollein

130

Matthias Niessner @niessner.bsky.social · 17/09/2025

Can we use video diffusion to generate 3D scenes? 𝐖𝐨𝐫𝐥𝐝𝐄𝐱𝐩𝐥𝐨𝐫𝐞𝐫 (#SIGGRAPHAsia25) creates fully-navigable scenes via autoregressive video generation. Text input -> 3DGS scene output & interactive rendering! 🌍http://mschneider456.github.io/world-explorer/ 📽️https://youtu.be/N6NJsNyiv6I

2162

Matthias Niessner @niessner.bsky.social · 05/08/2025

We further propose a color-based densification and progressive training scheme for improved quality and faster convergence. shivangi-aneja.github.io/projects/sca... youtu.be/VyWkgsGdbkk Great work by Shivangi Aneja, Sebastian Weiss, Irene Baeza Rojo, Prashanth Chandran, Gaspard Zoss, Derek Bradley

shivangi-aneja.github.io

ScaffoldAvatar: High-Fidelity Gaussian Avatars with Patch Expressions

000

Matthias Niessner @niessner.bsky.social · 05/08/2025

We operate on patch-based local expression features and increase the representation capacity by synthesizing 3D Gaussians dynamically by leveraging tiny scaffold MLPs conditioned on localized expressions.

shivangi-aneja.github.io

ScaffoldAvatar: High-Fidelity Gaussian Avatars with Patch Expressions

100

Matthias Niessner @niessner.bsky.social · 05/08/2025

ScaffoldAvatar: High-Fidelity Gaussian Avatars with Patch Expressions (#SIGGRAPH) We reconstruct ultra-high fidelity photorealistic 3D avatars capable of generating realistic and high-quality animations including freckles and other fine facial details. shivangi-aneja.github.io/projects/sca...

160

Matthias Niessner @niessner.bsky.social · 04/07/2025

TL;DR RGB-D scan as input -> compact, CAD scene representation that also features materials in order to create a digital copy that features the looks of a real environment. Great work by Zhening (Jack) Huang in collaboration with Xiaoyang Wu, Fangcheng Zhong, Hengshuang Zhao, Joan Lasenby

010

Matthias Niessner @niessner.bsky.social · 04/07/2025

📢 LiteReality: Graphics-Ready 3D Scene Reconstruction from RGB-D Scans🏠✨ -> converts RGB-D scans into compact, realistic, and interactive 3D scenes — featuring high-quality meshes, PBR materials, and articulated objects. 📷https://youtu.be/ecK9m3LXg2c 🌍https://litereality.github.io

1110

Matthias Niessner @niessner.bsky.social · 27/06/2025

Seven papers accepted at #ICCV2025! Exciting topics: lots of generative AI using transformers, diffusion, 3DGS, etc. focusing on image synthesis, geometry generation, avatars, and much more - check it out! So proud of everyone involved - let's go🚀🚀🚀 niessnerlab.org/publications...

162

Matthias Niessner @niessner.bsky.social · 21/06/2025

Want to work on cutting-edge #AI? We have several fully-funded 𝐏𝐡𝐃 & 𝐏𝐨𝐬𝐭𝐃𝐨𝐜 𝐨𝐩𝐞𝐧𝐢𝐧𝐠𝐬 in our Visual Computing & AI Lab in Munich! Apply here: application.vc.in.tum.de Topics have a strong focus on Generative AI, 3DGs, NeRFs, Diffusion, LLMs, etc.

1113

Matthias Niessner @niessner.bsky.social · 10/06/2025

#CVPR submissions per year have significantly increased. Now over 11k / year with an expectation to grow even further. This comes with a lot of implications, how to handle the reviews, presentations, etc. Kudos to the organizers for all the efforts that went into it.

051

Matthias Niessner @niessner.bsky.social · 10/06/2025

Super excited to be in Nashville for #CVPR2025! Looking forward to catching up with everyone -- feel free to reach out if you want to chat! Everything is a Honky Tonk :)

1130

Matthias Niessner @niessner.bsky.social · 09/06/2025

In addition, we introduce a new OLAT dataset of human heads that features high-resolution and high frame rate multi-view recordings of diverse subjects in a calibrated light stage setting. Great work by Jonathan Schmidt and Simon Giebenhain.

040

Matthias Niessner @niessner.bsky.social · 09/06/2025

📢BecomingLit: Relightable Gaussian Avatars with Hybrid Neural Shading📢 We propose a hybrid neural shading scheme for creating intrinsically decomposed 3DGS head avatars, that allow real-time relighting and animation. 🌍https://lnkd.in/evNt8bV2 📷https://lnkd.in/ekB5QeEK

1121

Matthias Niessner @niessner.bsky.social · 05/06/2025

📢Code Release of Pixel3DMM 📢 Looking for a robust and accurate face tracker? We handle challenging in-the-wild settings, such as extreme lighting conditions, fast movements, and occlusions. 👨‍💻https://lnkd.in/e3dX23WV 🌍https://lnkd.in/eQ3Zpn3J Pixel3DMM can be run on videos and single images.

2232

Matthias Niessner @niessner.bsky.social · 04/06/2025

📢PBR-SR: Mesh PBR Texture Super Resolution from 2D Image Priors📢 We propose a new optimization to up-sample textures of 3D assets (albedo, roughness, metallic, and normal maps) by leveraging 2D super-resolution models. 📝http://arxiv.org/abs/2506.02846 📽️https://youtu.be/eaM5S3Mt1RM

0132

Matthias Niessner @niessner.bsky.social · 27/05/2025

🚀🚀🚀Announcing our $13M funding round to build the next generation of AI: 𝐒𝐩𝐚𝐭𝐢𝐚𝐥 𝐅𝐨𝐮𝐧𝐝𝐚𝐭𝐢𝐨𝐧 𝐌𝐨𝐝𝐞𝐥𝐬 that can generate entire 3D environments anchored in space & time. 🚀🚀🚀 Interested? Join our world-class team: 🌍 spaitial.ai youtu.be/FiGX82RUz8U

youtu.be

SpAItial AI: Building Spatial Foundation Models

YouTube video by SpAItial AI

5539

Matthias Niessner @niessner.bsky.social · 09/05/2025

of 3D hair strand reconstructions from real-world scans of 400 different people, featuring complicated hairstyles, such as ponytails and buns. 🌍 seva100.github.io/GeomHair 📷 youtu.be/h9vqTiFo9As Great work by Rachmadio L., Artem Sevastopolsky Egor Zakharov, Vanessa Sklyarova

seva100.github.io

GeomHair

GeomHair: Reconstruction of Hair Strands from Colorless 3D Scans

000

Matthias Niessner @niessner.bsky.social · 09/05/2025

We enhance the reconstruction with a diffusion prior trained on synthetic hair data and adapted to each scan using a tailored text prompt, allowing us to recover both simple and complex hairstyles without relying on color input. We also introduce Strands400, the largest publicly available dataset

seva100.github.io

GeomHair

GeomHair: Reconstruction of Hair Strands from Colorless 3D Scans

100

Matthias Niessner @niessner.bsky.social · 09/05/2025

📢GeomHair: Reconstruction of Hair Strands from Colorless 3D Scans📢 We reconstruct hair strands from colorless 3D scans by extracting orientation cues directly from the mesh surface geometry by finding local characteristic lines and from shaded renderings using a neural 2D line detector.

1100

Matthias Niessner @niessner.bsky.social · 02/05/2025

Works for both single image and videos! We also introduce a new 3D face reconstruction benchmark that evaluates both neutral and posed face geometry. 🌍 simongiebenhain.github.io/pixel3dmm 📷 youtu.be/BwxwEXJwUDc Great work by Simon Giebenhain, Tobias Kirschstein, Martin Rünz, Lourdes Agapito

simongiebenhain.github.io

Pixel3DMM: Versatile Screen-Space Priors for Single-Image 3D Face Reconstruction

020

Matthias Niessner @niessner.bsky.social · 02/05/2025

📢Pixel3DMM: Versatile Screen-Space Priors for Single-Image 3D Face Reconstruction📢 -> highly accurate face reconstruction by training powerful VITs via surface normals & UV-coordinates estimation. These cues from our 2D foundation model constrain the 3DMM parameters, achieving great accuracy.

183

Matthias Niessner @niessner.bsky.social · 25/04/2025

We show how it can be used to reconstruct photorealistic scenes, and introduce a corresponding differentiable CUDA rasterizer that enables real-time rendering. LinPrim achieves comparable image quality with fewer primitives, adding a practical polyhedral option. 🎥 youtu.be/NRRlmFZj5KQ

nicolasvonluetzow.github.io

LinPrim: Linear Primitives for Differentiable Volumetric Rendering

000

Matthias Niessner @niessner.bsky.social · 25/04/2025

📢 LinPrim: Linear Primitives for Differentiable Volumetric Rendering 📢 We use octahedra or tetrahedra as explicit as volumetric building blocks for gradient-based novel view synthesis - as an alternative to 3D Gaussians with discrete, bounded geometry. 🌍 nicolasvonluetzow.github.io/LinPrim/

1121

Matthias Niessner @niessner.bsky.social · 23/04/2025

first place -- and current generators are unable to densely sample the full solution space. So obviously it's hard to imagine a future without AI-assisted coding for fast development, but at critical points one still needs human intervention to make the right decisions.

010