iiinova — where your digital self keeps evolving

Michael Moor @michaelmoor.bsky.social · 13/04/2026

Instead of retraining / adapting reasoning models for every domain, we can plug in reward modules to steer reasoning toward higher reliability, which is needed especially in high-stakes settings. 14/14

010

Michael Moor @michaelmoor.bsky.social · 13/04/2026

Bigger picture: 🧠 reasoning models = general-purpose 📚 process reward agents = domain-specific grounding modules → PRA enables a decoupling of how we reason from what we know 13/

110

Michael Moor @michaelmoor.bsky.social · 13/04/2026

Results / Take-aways (more details in paper): ✅ Strong improvements on medical reasoning benchmarks ✅ Works across multiple frozen policy models ✅ Generalizes beyond the model it was paired with 12/

100

Michael Moor @michaelmoor.bsky.social · 13/04/2026

Why this matters: In knowledge-intensive domains, correctness is not merely logical consistency → it requires alignment with external knowledge. PRA proposes a way to bring this into the reasoning & reward loop 11/

110

Michael Moor @michaelmoor.bsky.social · 13/04/2026

This unlocks something important: 👉 Tree-search reasoning paired with grounded knowledge Instead of committing to one chain of thought, PRA can: → explore multiple paths → retrieve different evidence per step → evaluate them in real time 10/

100

Michael Moor @michaelmoor.bsky.social · 13/04/2026

Key idea: PRA turns PRMs into active agents that can: • Check each reasoning step dynamically • Query external knowledge (guidelines, textbooks, etc.) • Provide immediate rewards during generation 9/

100

Michael Moor @michaelmoor.bsky.social · 13/04/2026

Here, we introduce Process Reward Agents (PRA). A framework that uses PRMs online to search, reward, and guide reasoning as it unfolds. 8/

100

Michael Moor @michaelmoor.bsky.social · 13/04/2026

Grounded (i.e. retrieval-augmented) PRMs typically operate post-hoc: They score reasoning only after the full trace is generated. This means: ❌ no real-time feedback ❌ no flexible search (e.g. tree exploration) ❌ limited ability to steer reasoning as it happens 7/

100

Michael Moor @michaelmoor.bsky.social · 13/04/2026

Recent work combines PRMs with retrieval (e.g. Med-PRM): - pull in external knowledge - critique reasoning traces step-by-step using external sources But there’s a catch 👇 6/

100

Michael Moor @michaelmoor.bsky.social · 13/04/2026

One promising direction to go beyond final answers: Process Reward Models (PRMs) Instead of only judging final answers, PRMs evaluate intermediate reasoning steps. 5/

100

Michael Moor @michaelmoor.bsky.social · 13/04/2026

This matters a lot. In domains like medicine, LLM reasoning is not only about the final answer/decision - we urgently need sound and defensible justifications along the way! 4/

100

Michael Moor @michaelmoor.bsky.social · 13/04/2026

But in knowledge-intensive domains: - step correctness often depends on external knowledge (consensus, guidelines, textbooks, local constraints etc.) spread across various sources - individual steps may not be easily verifiable in isolation 3/

100

Michael Moor @michaelmoor.bsky.social · 13/04/2026

In math/code, intermediate steps are often locally verifiable → you can +- easily verify if a step is correct (e.g. formal rules, symbolic solvers, code compilation & execution etc.) 2/

100

Michael Moor @michaelmoor.bsky.social · 13/04/2026

Preprint: arxiv.org/abs/2604.09482 Page: process-reward-agents.github.io Code: github.com/eth-medical-... Big thanks to a stellar team of co-authors: Jiwoong Sohn, Tomasz Sternal, Kenneth Styppa, and Torsten Hoefler! @ethz.ch 1/

arxiv.org

Process Reward Agents for Steering Knowledge-Intensive Reasoning

Reasoning in knowledge-intensive domains remains challenging as intermediate steps are often not locally verifiable: unlike math or code, evaluating step correctness may require synthesizing clues acr...

100

Michael Moor @michaelmoor.bsky.social · 13/04/2026

[Preprint Alert] 𝐏𝐑𝐎𝐂𝐄𝐒𝐒 𝐑𝐄𝐖𝐀𝐑𝐃 𝐀𝐆𝐄𝐍𝐓𝐒 (𝐏𝐑𝐀) Why is it relatively easy to get LLMs to produce strong reasoning traces in math/code… but much harder in application domains like health? And what can we do against it? Check out our new paper & 🧵below:

210

Michael Moor @michaelmoor.bsky.social · 19/06/2025

Check out Med-PRM, an approach for LLMs to verify each reasoning step against guidelines: 🔗 Page: med-prm.github.io 📄 Paper: arxiv.org/abs/2506.11474 🧠 Model: huggingface.co/dmis-lab/lla... 📚 Dataset: huggingface.co/datasets/dmi... 💻 Code: github.com/eth-medical-... 🧵 Thread: tinyurl.com/yu933dx6

030

Michael Moor @michaelmoor.bsky.social · 30/04/2025

Welcome to our new lab page 🚀 bsse.ethz.ch/mail

030

Michael Moor @michaelmoor.bsky.social · 24/03/2025

#AI agent labs are becoming better at producing autonomous research. Still, they operate in isolation w/o improving & interacting. Here, we introduce 𝐀𝐠𝐞𝐧𝐭𝐑𝐱𝐢𝐯, where agent laboratories can upload & download latest research - which accelerates their progress: Great effort led by Samuel Schmidgall!

031

Reposted by Michael Moor

D-BSSE, ETH Zurich @handle.invalid · 10/03/2025

Today, our two new faculty members held their inaugural lectures @ethzurich.bsky.social. Basile Wicky | Biomedical Design Lab, presented on designing proteins that interface with life; Michael Moor @michaelmoor.bsky.social | Medical AI Lab, spoke about AI in medicine. Recordings > u.ethz.ch/hQdQl

032

Reposted by Michael Moor

Sebastian Schönherr @sebschoenherr.bsky.social · 06/03/2025

Today @michaelmoor.bsky.social took a train from Zurich to Innsbruck to kick-off our new Faculty of "AI and Scientific Computing"! 🎉 He talked about LLMs and Medical AI Agents. Exciting science and great discussions! Thanks! More info about our faculty: aiscm.i-med.ac.at #ai #scientificcomputing

aiscm.i-med.ac.at

Faculty of AI and Scientific Computing in Medicine (AISCM) Medical University of Innsbruck

Faculty of AI and Scientific Computing in Medicine

021

Michael Moor @michaelmoor.bsky.social · 16/12/2024

Let's get paper sharing started here! I'll start: Interesting new preprint on Multimodal medical preference optimization: arxiv.org/pdf/2412.06141 @huaxiuyaoml.bsky.social (and others)

arxiv.org

010

Reposted by Michael Moor

Charlotte Bunne @bunnech.bsky.social · 12/12/2024

How can we build an Al virtual cell that simulates all functions and interactions of a cell? How will it transform research and drive breakthroughs in programmable biology, drug discovery and personalized medicine? Take a look at our paper in @cellpress.bsky.social! www.cell.com/cell/fulltex...

39021

Michael Moor @michaelmoor.bsky.social · 08/12/2024

UI / UX suggestion for #bluesky: I would remove the ".bsky.social" string that clutters the app. Like when looking at a list of n accounts, one has to visually ignore this suffix n times. I suspect that small UI things like this could make a big impact in getting more momentum.

020

Michael Moor @michaelmoor.bsky.social · 04/12/2024

OpenAI coming to Switzerland! Congrats on the new roles!

050