iiinova — where your digital self keeps evolving

Hadi Sotoudeh

@mhsotoudeh.bsky.social

48 followers 179 following 9 posts

AI4Science PhD @ Cambridge | PolymathicAI @polymathicai.bsky.social | ✨🌌💫

Posts Replies Media

Hadi Sotoudeh @mhsotoudeh.bsky.social · 23/04/2026

If you're at ICLR and curious about physics foundation models, come by: 🗓️ Sunday, April 26 📍 Room 201 A+B 🕘 9:45–10:35 A pleasure to work on this with @payelm.bsky.social, @rubenohana.bsky.social, @michael-mccabe.bsky.social, @lawrennd.bsky.social, Shirley Ho, and @milescranmer.bsky.social. 6/6

Closing slide with paper title, author list, links for paper and poster, and invitation to visit the work at the ICLR 2026 AI & PDE Workshop.

043

Hadi Sotoudeh @mhsotoudeh.bsky.social · 23/04/2026

Additional insight: A mostly frozen in-domain tokeniser improved long-horizon autoregressive rollouts, suggesting a regularising effect. It also reduced trainable parameters from 5M to 85k. 5/6

Additional insight slide on freezing strategy, showing that keeping most of an in-domain pretrained tokeniser frozen improves long-horizon autoregressive rollouts, suggesting a regularising effect. Trainable parameters drop from 5 million to 85 thousand.

100

Hadi Sotoudeh @mhsotoudeh.bsky.social · 23/04/2026

Main result: In-domain tokeniser pretraining gave the strongest gains. After 10k steps, error was 64% lower than training from scratch. Out-of-domain pretraining also helped, but less. 4/6

Main result slide showing that tokeniser pretraining improves emulation performance. In-domain pretraining gives the best results, reducing Variance-normalised Root Mean Squared Error (VRMSE) by 64% after 10k steps, while out-of-domain pretraining reduces error by 19% relative to no pretraining.

120

Hadi Sotoudeh @mhsotoudeh.bsky.social · 23/04/2026

How we tested it: For an emulation task, we compared three training settings: • no pretraining • in-domain pretraining • out-of-domain pretraining Using datasets from The Well collection (polymathic-ai.org/the_well) 3/6

Experimental setup slide comparing three training settings for emulation: no pretraining, in-domain tokeniser pretraining on the same dataset as emulator, and out-of-domain pretraining on a different dataset. Uses datasets from The Well collection.

100

Hadi Sotoudeh @mhsotoudeh.bsky.social · 23/04/2026

Why consider pretraining? Physics foundation models are asked to learn two tasks at once: • encode high-resolution spatiotemporal data • model the underlying dynamics In many other ML domains, the encoding task is often given a head start through pretraining. We wanted to test that idea here. 2/6

Motivation slide explaining that physics foundation models must learn two tasks jointly: encode high-resolution spatiotemporal data and model dynamics. It asks whether pretraining the tokeniser could make models faster or more accurate.

100

Hadi Sotoudeh @mhsotoudeh.bsky.social · 23/04/2026

Physics foundation models are usually trained end-to-end from scratch. But is that the best way? In new work from @polymathicai.bsky.social, we study tokeniser pretraining. Presenting at @iclr-conf.bsky.social AI & PDE. 📄 arxiv.org/abs/2603.05598 #ICLR2026 #AIforScience #FoundationModels 1/6

Title slide asking whether physics foundation models should pretrain their tokeniser. Diagram compares two approaches: joint training from scratch versus two-stage training with tokeniser pretraining followed by foundation model training.

110

Hadi Sotoudeh @mhsotoudeh.bsky.social · 22/07/2025

It was a huge honor to give a keynote at the ML4Astro workshop in Vancouver, sharing our journey at @polymathicai.bsky.social toward building generalist AI models for science. Even more special to share the stage with my friend @astronolan.bsky.social – grateful for the chance to do this together!

Hadi Sotoudeh and Nolan Koblischke stand at the front of a lecture hall delivering a keynote at the ML4Astro workshop in Vancouver. The slide projected behind them reads, "How do we build a generalist AI model for science?" and outlines challenges along a spectrum from structured data to language-like data. Several attendees are seated, some with laptops open.

010

Hadi Sotoudeh @mhsotoudeh.bsky.social · 03/12/2024

Feel free to: 🪜🕳️ Dip into "The Well": github.com/PolymathicAI... ✨🌌 Explore the "Mutimodal Universe": github.com/MultimodalUn... #AI4Science #FoundationModels #DemocratisingAI

010

Hadi Sotoudeh @mhsotoudeh.bsky.social · 03/12/2024

What an era to be a scientist! By preparing "teaching material" for some of the most intelligent mathematical models, @polymathicai.bsky.social is taking one step closer to unlocking entirely new frontiers in scientific discovery. Very proud of our team and excited about the adventures ahead!! 👀

120