iiinova — where your digital self keeps evolving

Thaddäus Wiedemer @thwiedemer.bsky.social · 13/10/2026

Robert Geirhos and I are #hiring student researchers at Google DeepMind for 2027. Express your interest here: docs.google.com/forms/d/e/1F... - full-time for 6 months - on-site in Zurich - co-advised by Robert Geirhos and me - continuing our work on video models for visual intelligence

docs.google.com

Interest for Google DeepMind Zurich 2027 Student Researcher position on emergent capabilities of video models

Please complete this form to express your interest for a Google DeepMind student researcher position on emergent capabilities of video models, located in Zurich. Project description: This project inv...

000

Thaddäus Wiedemer @thwiedemer.bsky.social · 30/07/2026

Work with Robert Geirhos, Effie Li, @nehakalibhat.bsky.social, Zi Wang, Mani Malek, Oyvind Tafjord, Kevin Swersky, Been Kim, and Priyank Jaini. Many fun videos on our project page: visual-prompt-engineering.github.io

visual-prompt-engineering.github.io

Visual Prompt Engineering for Video Models

Visual prompt engineering (VIPE) improves video reasoning performance across tasks by transforming the input task image.

000

Thaddäus Wiedemer @thwiedemer.bsky.social · 30/07/2026

An important different read of these results is that many visual reasoning benchmarks (which typically are abstract sketches) systematically underestimate video model capabilities.

100

Thaddäus Wiedemer @thwiedemer.bsky.social · 30/07/2026

Although realism isn't the only knob to turn: visual prompt engineering can also increase contrast, reduce ambiguity, or improve affordances in the scene for actions specified in the prompt.

100

Thaddäus Wiedemer @thwiedemer.bsky.social · 30/07/2026

So why does it work? A simple experiment shows that increased realism helps improving generation consistency, potentially because it addresses a distribution shift from largely realistic training data.

100

Thaddäus Wiedemer @thwiedemer.bsky.social · 30/07/2026

On some tasks, visual prompt engineering can be more effective than text prompt engineering, although there doesn't seem to be a general rule.

100

Thaddäus Wiedemer @thwiedemer.bsky.social · 30/07/2026

We don't even need a good heuristic to pick a best variant: A simple majority vote can turn visual prompt engineering into a test-time scaling strategy. It's more effective to try one video each for many visual variants, than to keep generating videos for the original prompt.

100

Thaddäus Wiedemer @thwiedemer.bsky.social · 30/07/2026

For many tasks, even a naive automated pipeline can find some visual prompt variants that drastically reduce errors.

100

Thaddäus Wiedemer @thwiedemer.bsky.social · 30/07/2026

A good visual prompt (here, abstract sketch -> realistic scene) improves reasoning performance for many models. The ablation shows it's more about visual realism than just 3D cues from shadows and perspective.

100

Thaddäus Wiedemer @thwiedemer.bsky.social · 30/07/2026

When an LLM is underperforming, we tweak the prompt. Why not do the same for video models solving visual reasoning tasks? Visual prompt engineering changes the start frame appearance but not the task, improving performance. 📄 arxiv.org/pdf/2607.25537 🌐 visual-prompt-engineering.github.io

100

Thaddäus Wiedemer @thwiedemer.bsky.social · 03/02/2026

This work was a great collaboration; special shout-out to @jana-z.bsky.social for leading this project and submitting the first paper of her PhD!

000

Thaddäus Wiedemer @thwiedemer.bsky.social · 03/02/2026

Whether self-generated visuals can at some point serve a function similar to mental imagery in human thought remains to be seen. For now, MentisOculi provides a small suite of tasks to study this topic.

100

Thaddäus Wiedemer @thwiedemer.bsky.social · 03/02/2026

The fact that we don't see strong benefits of using even ground-truth visuals points to information in the visual/textual domains being somewhat misaligned, potentially because models are not trained for similar tasks.

100

Thaddäus Wiedemer @thwiedemer.bsky.social · 03/02/2026

This work is motivated by the same intuition as my work on Video models last fall: Can media generation capabilities be useful beyond just generating nice visuals? For real-world, embodied applications being able to visualize the outcome of an action seems useful.

100

Thaddäus Wiedemer @thwiedemer.bsky.social · 03/02/2026

How useful are self-generated 'mental images' (visual aids) in MLLM/UMM reasoning? Turns out: currently not very. Visualizations have small errors that compound in multi-step problems, and models often ignore correct visual aids in their decision making.

121

Reposted by Thaddäus Wiedemer

Wieland Brendel @wielandbrendel.bsky.social · 15/12/2025

🚀 We're hiring! The @ellisinsttue.bsky.social leads the AI development for Germany’s new open-source nationwide Adaptive Intelligent System learning platform for schools (as part of a consortium led by Assecor & KI macht Schule, and mandated by the FWU). 👉 Apply now: forms.gle/XmLkwEDD45fY...

263

Reposted by Thaddäus Wiedemer

A. Sophia Koepke @askoepke.bsky.social · 21/10/2025

🎉 Excited to present our paper VGGSounder: Audio‑Visual Evaluations for Foundation Models today at #ICCV2025! 🕦 Poster Session 1 | 11:30–13:30 📍 Poster #88 Come by if you're into audio-visual learning and want to know whether multiple modalities actually help or hurt.

161

Thaddäus Wiedemer @thwiedemer.bsky.social · 25/09/2025

I'm truly honored to have worked on this at Google DeepMind with my amazing collaborators! With 2 months left in my internship, I'm excited about our next steps in this direction!

010

Thaddäus Wiedemer @thwiedemer.bsky.social · 25/09/2025

And as with other 'zero-shot' works, it's clear that Veo has been exposed to samples of many of our tasks in the training data. The promise lies into its ability to quickly be adapted for general tasks with just a prompt, no fine-tuning required!

110

Thaddäus Wiedemer @thwiedemer.bsky.social · 25/09/2025

Of course, performance is not perfect yet and lacks behind SotA. Video models are also expensive to train and run, so they won't replace all vision models just yet. But the rapid progress from Veo 2 to Veo 3 illustrates their potential to become vision foundation models.

100

Thaddäus Wiedemer @thwiedemer.bsky.social · 25/09/2025

Intuitively, some tasks are easier to directly solve in the vision domain, and we also observe this in maze solving tasks. This makes me super excited about a future where generalist vision and language models could be integrated for reasoning in the real world by 'imagining' possible outcomes.

100

Thaddäus Wiedemer @thwiedemer.bsky.social · 25/09/2025

On the reasoning side, videos as 'chain-of-frames' parallel chain-of-thought in LLMs. Complex visual tasks that an image editing model like Nano Banana would have to solve in one go can be broken down into smaller steps.

100

Thaddäus Wiedemer @thwiedemer.bsky.social · 25/09/2025

Specifically, Veo 3 can perceive (segment, locacalize, detect edges, ...), model (physics, abstract relations, memory), manipulate (edit images, simulate robotics), and reason about the visual world. Video models might well become vision foundation models.

100

Thaddäus Wiedemer @thwiedemer.bsky.social · 25/09/2025

Are we experiencing a 'GPT moment' in vision? In our new preprint, we show that generative video models can solve a wide range of tasks across the entire vision stack without being explicitly trained for it. 🌐 video-zero-shot.github.io 1/n

241

Thaddäus Wiedemer @thwiedemer.bsky.social · 18/02/2025

Check out our newest paper! As always, it was super fun working on this with @prasannamayil.bsky.social

051

Reposted by Thaddäus Wiedemer

Andreas Hochlehnert @ahochlehnert.bsky.social · 17/02/2025

CuratedThoughts: Data Curation for RL Datasets 🚀 Since DeepSeek-R1 introduced reasoning-based RL, datasets like Open-R1 & OpenThoughts emerged for fine-tuning & GRPO. Our deep dive found major flaws — 25% of OpenThoughts needed elimination by data curation. Here's why 👇🧵

1139

Reposted by Thaddäus Wiedemer

Wieland Brendel @wielandbrendel.bsky.social · 11/02/2025

🚀 We’re hiring! Join Bernhard Schölkopf & me at @ellisinsttue.bsky.social to push the frontier of #AI in education! We’re building cutting-edge, open-source AI tutoring models for high-quality, adaptive learning for all pupils with support from the Hector Foundation. 👉 forms.gle/sxvXbJhZSccr...

21014