iiinova — where your digital self keeps evolving

Daniel Palenicek @daniel-palenicek.bsky.social · 22/04/2026

Headed to Rio for #ICLR 🇧🇷 come say hi at our poster! XQC: a principled look at critic optimization. BN+WN+cross-entropy → condition numbers orders of magnitude smaller than baselines. SOTA on 70 continuous ctrl tasks w/ 4.5× less params. 📅 Thu, 1030–1300 📍 Pavilion 4, #4518

031

Daniel Palenicek @daniel-palenicek.bsky.social · 03/02/2026

Hey @araffin.bsky.social sorry, I missed your question. "Coming soon" was a but optimistic, I got caught up in other projected which I did not expect... We will release the XQC code in the next few weeks! We won't do a seperate CrossQ+WN code release though.

020

Daniel Palenicek @daniel-palenicek.bsky.social · 03/02/2026

🎉 Really excited, our paper "XQC: Well-conditioned Optimization Accelerates Deep Reinforcement Learning" has been accepted at #ICLR2026. If you are interested in reinforcement learning, sample-efficiency, compute-efficiency go check it out. See you in Rio!

0103

Daniel Palenicek @daniel-palenicek.bsky.social · 13/12/2025

I'm super excited to have been named an #NVIDIA Graduate Fellowship Finalist! 💚 Huge thanks to my supervisor @jan-peters.bsky.social and all my collaborators. Can't wait to join the NVIDIA Seattle Robotics Lab for my internship next summer! 🤖 blogs.nvidia.com/blog/graduat...

blogs.nvidia.com

NVIDIA Awards up to $60,000 Research Fellowships to PhD Students

The Graduate Fellowship Program announced the latest awards of up to $60,000 each to 10 Ph.D. students involved in research that spans all areas of computing innovation.

063

Daniel Palenicek @daniel-palenicek.bsky.social · 06/12/2025

Had a really great time presenting our #NeurIPS paper at the poster session today. Thanks to everyone who stopped by. If you are interested in sample-efficient #RL, check out our work: Scaling Off-Policy Reinforcement Learning with Batch and Weight Normalization

141

Daniel Palenicek @daniel-palenicek.bsky.social · 02/10/2025

Read the full preprint here: 👉 arxiv.org/pdf/2509.25174 Code coming soon. We’d love feedback & discussion! 💬

arxiv.org

010

Daniel Palenicek @daniel-palenicek.bsky.social · 02/10/2025

Key takeaway: Well-conditioned optimization > raw scale. XQC proves principled architecture choices can outperform larger, more complex ones

110

Daniel Palenicek @daniel-palenicek.bsky.social · 02/10/2025

📊 Results across 70 tasks (55 proprioception + 15 vision-based): ⚡️ Matches/outperforms SimbaV2, BRO, BRC, MRQ, and DRQ-V2 🌿~4.5× fewer parameters and 1/10 FLOP/s of SimbaV2 💪Especially strong on the hardest tasks: HumanoidBench, DMC Hard & DMC Humanoids from pixels

100

Daniel Palenicek @daniel-palenicek.bsky.social · 02/10/2025

This leads to XQC, a streamlined extension of Soft Actor-Critic with ✅ only 4 hidden layers ✅ BN after each linear layer ✅ WN projection ✅ CE critic loss Simplicity + principled design = efficiency ⚡️

100

Daniel Palenicek @daniel-palenicek.bsky.social · 02/10/2025

🔑 Insight: A simple synergy—BatchNorm + WeightNorm + Cross-Entropy loss—makes critics dramatically more well-conditioned. ➡️Result: Stable effective learning rates and smoother optimization.

110

Daniel Palenicek @daniel-palenicek.bsky.social · 02/10/2025

Instead of "bigger is better," we ask: Can better conditioning beat scaling? By analyzing the Hessian eigenspectrum of critic networks, we uncover how different architectural choices shape optimization landscapes.

100

Daniel Palenicek @daniel-palenicek.bsky.social · 02/10/2025

🚀 New preprint! Introducing XQC— a simple, well-conditioned actor-critic that achieves SOTA sample efficiency in #RL ✅ ~4.5× fewer parameters than SimbaV2 ✅ Scales to vision-based RL 👉 arxiv.org/pdf/2509.25174 Thanks to Florian Vogt @joemwatson.bsky.social @jan-peters.bsky.social

172

Daniel Palenicek @daniel-palenicek.bsky.social · 23/05/2025

If you're working on RL stability, plasticity, or sample efficiency, this might be relevant for you. We'd love to hear your thoughts and feedback! Come talk to us at RLDM in June in Dublin (rldm.org)

rldm.org

RLDM | The Multi-disciplinary Conference on Reinforcement Learning and Decision Making

100

Daniel Palenicek @daniel-palenicek.bsky.social · 23/05/2025

📚 TL;DR: We combine BN + WN in CrossQ for stable high-UTD training and SOTA performance on challenging RL benchmarks. No need for network resets, no critic ensembles, no other tricks... Simple regularization, big gains. Paper: t.co/Z6QrMxZaPY

t.co

https://arxiv.org/abs/2502.07523v2

100

Daniel Palenicek @daniel-palenicek.bsky.social · 23/05/2025

⚖️ Simpler ≠ Weaker: Compared to SOTA baselines like BRO our method: ✅ Needs 90% fewer parameters (~600K vs. 5M) ✅ Avoids parameter resets ✅ Scales stably with compute. We also compare strongly to the concurrent SIMBA algorithm. No tricks—just principled normalization. ✨

100

Daniel Palenicek @daniel-palenicek.bsky.social · 23/05/2025

🔬 The Result: CrossQ + WN scales reliably with increasing UTD—no more resets, no critic ensembles, no other tricks. We match or outperform SOTA on 25 continuous control tasks from DeepMind Control Suite & MyoSuite, including dog 🐕 and humanoid🧍‍♂️tasks across UTDS.

100

Daniel Palenicek @daniel-palenicek.bsky.social · 23/05/2025

➡️ With growing weight norm, the effective learning rate decreases, and learning slows down/stops. 💡Solution: After each gradient update, we rescale parameters to the unit sphere, preserving plasticity and keeping the effective learning rate stable.

100

Daniel Palenicek @daniel-palenicek.bsky.social · 23/05/2025

🧠Key Idea: BN improves sample efficiency, but fails to reliably scale with complex tasks & high UTDs due to growing weight norms. However, BN regularized networks are scale invariant w.r.t. their weights; yet, the gradient scales inversely proportional (Van Laarhoven 2017)

100

Daniel Palenicek @daniel-palenicek.bsky.social · 23/05/2025

🔍 Background: Off-policy RL methods like CrossQ (Bhatt* & Palenicek* et al. 2024) are sample-efficient but struggle to scale to high update-to-data (UTD) ratios. We identify why scaling CrossQ fails—and fix it with a surprisingly effective tweak: Weight Normalization (WN). 🏋️

100

Daniel Palenicek @daniel-palenicek.bsky.social · 23/05/2025

🚀 New preprint "Scaling Off-Policy Reinforcement Learning with Batch and Weight Normalization"🤖 We propose CrossQ+WN, a simple yet powerful off-policy RL for more sample-efficiency and scalability to higher update-to-data ratios. 🧵 t.co/Z6QrMxZaPY #RL @ias-tudarmstadt.bsky.social

t.co

https://arxiv.org/abs/2502.07523v2

171

Daniel Palenicek @daniel-palenicek.bsky.social · 19/03/2025

Check out our latest work, where we train an omnidirectional locomotion policy directly on a real quadruped robot in just a few minutes based on our CrossQ RL algorithm 🚀 Shoutout @nicobohlinger.bsky.social, Jonathan Kinzel. @ias-tudarmstadt.bsky.social @hessianai.bsky.social

020