iiinova — where your digital self keeps evolving

Herbie(Zizhan) He @herbiehe.bsky.social · 18/03/2026

[7/8] Beyond Grid World, ESWM is scalable to the more complex MiniGrid (high-dimensional observation) and 3D indoor scenes ProcThor (realistic pixel observations).

110

Herbie(Zizhan) He @herbiehe.bsky.social · 18/03/2026

[6/8] When environments change (e.g., new obstacles), ESWM adapts by updating its temporally and spatially independent memories. No retraining is needed.

110

Herbie(Zizhan) He @herbiehe.bsky.social · 18/03/2026

[5/8] ESWM also supports efficient exploration by acting on uncertainty to collect experiences and navigate between states.

110

Herbie(Zizhan) He @herbiehe.bsky.social · 18/03/2026

[4/8] In GridWorld experiments: 1) Transformer >> LSTM & Mamba. 2) ESWM generalizes to novel observations and structures. 3) Its latent space reflects the environment structure. 4) It predicts by integrating independent transitions.

111

Herbie(Zizhan) He @herbiehe.bsky.social · 18/03/2026

[3/8] ESWM is designed to operate on sets of temporally independent transitions. Given such a set, it infers unseen transitions. The model is meta-trained across environments to support generalization. We show three settings in which we validate ESWM.

110

Herbie(Zizhan) He @herbiehe.bsky.social · 18/03/2026

[2/8] In contrast, neuroscience evidence suggests that animals can build spatial representation across independent experiences (i.e day1: A->B, day2: B->C, day3: infers A->C). Motivated by these observations, we introduce ESWM:

120

Herbie(Zizhan) He @herbiehe.bsky.social · 18/03/2026

[1/8] Existing world models rely on a sequence of observations to predict future states. This leads to: 1) redundancy due to temporal overlap (contexts grow for large envs), 2) limited adaptability when environments change due to temporal dependency.

120

Herbie(Zizhan) He @herbiehe.bsky.social · 18/03/2026

New paper 🚨 #ICLR26 Most world models predict the future from a past trajectory. But neuroscience suggests that such inference can instead be made from temporally independent experiences. We built the Episodic Spatial World Model (ESWM), a model that does exactly this: Video abstract [1/2]

183

Reposted by Herbie(Zizhan) He

Pablo Samuel Castro @pcastr.bsky.social · 17/03/2026

New paper 🚨 "Stable Deep Reinforcement Learning via Isotropic Gaussian Representations" Deep RL suffers from unstable training, representation collapse, and neuron dormancy. We show that a simple geometric insight, isotropic Gaussian representations, can fix this. Here's how 👇

2295

Herbie(Zizhan) He @herbiehe.bsky.social · 28/06/2025

[11/12] 🔧 When environments change—say a new wall appears—ESWM adapts instantly. No retraining is needed. Just update the memory bank and the model replans. This separation of memory and reasoning makes ESWM highly flexible.

120

Herbie(Zizhan) He @herbiehe.bsky.social · 28/06/2025

[10/12] 🧭 It gets even better! ESWM can navigate between arbitrary points using only its memory bank—planning efficiently in latent space with near-optimal paths. No access to global maps or coordinates required.

120

Herbie(Zizhan) He @herbiehe.bsky.social · 28/06/2025

[9/12] 🚶 With no additional training, ESWM can explore novel environments efficiently by acting on uncertainty.

120

Herbie(Zizhan) He @herbiehe.bsky.social · 28/06/2025

[8/12] ⚙️ How are these maps built? We find that ESWM stitches together memories via overlapping states—merging local transitions into global structure. Obstacles and boundaries serve as spatial anchors, guiding how memories are organized in latent space.

231

Herbie(Zizhan) He @herbiehe.bsky.social · 28/06/2025

[7/12] 🏞️ How does ESWM solve the task? Using ISOMAP, we visualize its latent representations—beautifully organized spatial layouts emerge from its internal states, even when the model sees only a small part or out-of-distribution environments.

120

Herbie(Zizhan) He @herbiehe.bsky.social · 28/06/2025

[6/12] ⚡️ Transformer-based ESWM models outperform LSTMs and Mamba, especially in settings where observations are compositional. Attention allows the model to flexibly bind relevant memories and generalize across structures.

120

Herbie(Zizhan) He @herbiehe.bsky.social · 28/06/2025

[5/12] To train ESWM, we use meta-learning across diverse environments. At test time, the model gets a minimal set of disjoint episodic memories (single transitions) and must predict a missing element in a new transition—without ever seeing the full map.

120

Herbie(Zizhan) He @herbiehe.bsky.social · 28/06/2025

[4/12] 🧠 Inspired by the MTL’s architecture and function, we built ESWM: a neural network that infers the structure of its environment from isolated, one-step transitions—just like the brain integrates episodes into a cognitive map.

121

Herbie(Zizhan) He @herbiehe.bsky.social · 28/06/2025

[3/12] Such shortcut-seeking behavior is supported by internal world models in the brain’s medial temporal lobe (MTL), an area involved in both episodic memory and spatial navigation.

120

Herbie(Zizhan) He @herbiehe.bsky.social · 28/06/2025

🧠 Can a neural network build a spatial map from scattered episodic experiences like humans do? We introduce the Episodic Spatial World Model (ESWM)—a model that constructs flexible internal world models from sparse, disjoint memories. 🧵👇 [1/12]

3202