iiinova — where your digital self keeps evolving

DurstewitzLab @durstewitzlab.bsky.social · 04/10/2026

In our #NeurIPS2026 paper “A Minimal Interpretable Architecture for Zero-Shot Reconstruction of Dynamical Systems” (preprint: arxiv.org/abs/2607.14937) we reduce a DS FM to ingredients minimally necessary to reproduce long-term stat. and geom. properties of DS, even outperforming many TS & DS FM.

0143

DurstewitzLab @durstewitzlab.bsky.social · 02/10/2026

In our #NeurIPS2026 paper “Topological Out-of-Domain Generalization in Dynamical Systems (DS) Reconstruction” (arxiv.org/abs/2606.22969) we aim to infer the DS generating observed TS jointly with control parameters, in order to extrapolate to different dynamical regimes, e.g. across tipping points.

0103

DurstewitzLab @durstewitzlab.bsky.social · 01/10/2026

In our #NeurIPS2026 spotlight “Parallel-in-Time Training of Recurrent Neural Networks for Dynamical Systems (DS) Reconstruction” (preprint: arxiv.org/abs/2605.12683) we speed up training of nonlinear RNNs on time series from chaotic DS by >100x by combining DEER with generalized teacher forcing.

1154

DurstewitzLab @durstewitzlab.bsky.social · 31/08/2026

Next week I will talk about *foundation models for dynamical systems reconstruction* at ECML in Naples, ml4its.github.io/ml4its2026/. I will cover some of the latest from the group, like efficient parallel-in-time training for DSR and minimal mechanisms for zero-shot DSR.

050

DurstewitzLab @durstewitzlab.bsky.social · 10/05/2026

In a #ICML2026 position paper we argue a dynamical systems perspective is needed to drive time series models forward: arxiv.org/abs/2602.16864 For TS, we need to move away from transformers that do not respect a system’s dynamical structure, esp. if out-of-domain generalization & insight is sought.

05213

DurstewitzLab @durstewitzlab.bsky.social · 10/05/2026

Neural ODEs are great as continuous-time dynamical systems, but slow and tedious to train. In a new #ICML2026 paper we intro a novel solver for continuous-time RNNs that does not rely on numerical integration. It's not only way faster & robust, but enables explicit analysis: arxiv.org/abs/2602.15649

0192

DurstewitzLab @durstewitzlab.bsky.social · 01/03/2026

In a new #ICLR2026 paper we provide an algorithm for semi-analytically constructing un-/stable manifolds of fixed points and cycles of ReLU-based RNNs: openreview.net/pdf?id=EAwLA... These manifolds provide a skeleton for the system’s dynamics, dissecting the state space into basins of attraction.

0132

DurstewitzLab @durstewitzlab.bsky.social · 05/12/2025

Tomorrow Christoph will present DynaMix, the first foundation model for dynamical systems reconstruction, at #NeurIPS2025 Exhibit Hall C,D,E #2303

0113

DurstewitzLab @durstewitzlab.bsky.social · 15/08/2025

We have openings for several fully-funded positions (PhD & PostDoc) at the intersection of AI/ML, dynamical systems, and neuroscience within a BMFTR-funded Neuro-AI consortium, at Heidelberg University & Central Institute of Mental Health: www.einzigartigwir.de/en/job-offer... More info below ...

171

DurstewitzLab @durstewitzlab.bsky.social · 13/07/2025

Got prov. approval for 2 major grants in Neuro-AI & Dynamical Systems Reconstruction, on learning & inference in non-stationary environments, out-of-domain generalization, and DS foundation models. To all AI/math/DS enthusiasts: Expect job announcements (PhD/PostDoc) soon! Feel free to get in touch.

0348

DurstewitzLab @durstewitzlab.bsky.social · 19/06/2025

Just heading back from a fantastic workshop on neural dynamics at Gatsby/ London, organized by Tatiana Engel, Bruno Averbeck, & Peter Latham. Enjoyed seeing so many old friends, Memming Park, Carlos Brody, Wulfram Gerstner, Nicolas Brunel & many others … Discussed our recent DS foundation models …

060

DurstewitzLab @durstewitzlab.bsky.social · 20/05/2025

We dive a bit into the reasons why current time series FMs not trained for DS reconstruction fail, and conclude that a DS perspective on time series forecasting & models may help to advance the #TimeSeriesAnalysis field. (6/6)

010

DurstewitzLab @durstewitzlab.bsky.social · 20/05/2025

Remarkably, it not only generalizes zero-shot to novel DS, but it can even generalize to new initial conditions and regions of state space not covered by the in-context information. (5/6)

110

DurstewitzLab @durstewitzlab.bsky.social · 20/05/2025

And no, it’s neither based on Transformers nor Mamba – it’s a new type of mixture-of-experts architecture based on the recently introduced AL-RNN (proceedings.neurips.cc/paper_files/...), specifically trained for DS reconstruction. #AI (4/6)

100

DurstewitzLab @durstewitzlab.bsky.social · 20/05/2025

It often even outperforms TS FMs on forecasting diverse empirical time series, like weather, traffic, or medical data, typically used to train TS FMs. This is surprising, cos DynaMix’ training corpus consists *solely* of simulated limit cycles & chaotic systems, no empirical data at all! (3/6)

100

DurstewitzLab @durstewitzlab.bsky.social · 20/05/2025

Unlike TS FMs, DynaMix exhibits #ZeroShotLearning of long-term stats of unseen DS, incl. attractor geometry & power spectrum, w/o *any* re-training, just from a context signal. It does so with only 0.1% of the parameters of Chronos & 10x faster inference times than the closest competitor. (2/6)

100

DurstewitzLab @durstewitzlab.bsky.social · 20/05/2025

Can time series (TS) #FoundationModels (FM) like Chronos zero-shot generalize to unseen #DynamicalSystems (DS)? No, they cannot! But *DynaMix* can, the first TS/DS FM based on principles of DS reconstruction, capturing the long-term evolution of out-of-domain DS: arxiv.org/pdf/2505.131... (1/6)

1113

DurstewitzLab @durstewitzlab.bsky.social · 26/01/2025

This gives rise to an interpretable latent feature space, where datasets with similar dynamics cluster. Intriguingly, this clustering according to *dynamical systems features* led to much better separation of groups than could be achieved by more trad. time series features. (3/4)

140

DurstewitzLab @durstewitzlab.bsky.social · 26/01/2025

We show applications like transfer & few-shot learning, but most interestingly perhaps, subject/system-specific features were often linearly related to control parameters of the underlying dynamical system trained on … (2/4)

120

DurstewitzLab @durstewitzlab.bsky.social · 26/01/2025

Toward interpretable #AI foundation models for #DynamicalSystems reconstruction: Our paper on transfer & few-shot learning for dynamical systems just got accepted for #ICLR2025 ! Previous version: arxiv.org/pdf/2410.04814; strongly updated version will be available soon ... (1/4)

1122

DurstewitzLab @durstewitzlab.bsky.social · 25/12/2024

That paper discusses an important issue for RNNs as used in neurosci. But we would argue that many RNN approaches do not truly reconstruct DS, for which we demand also agreement in long-term stats, attractor geometry, and generative perform. (esp. in chaotic systems, MSE as stats can be misleading).

2152

DurstewitzLab @durstewitzlab.bsky.social · 24/12/2024

In proceedings.neurips.cc/paper_files/... we provided a highly efficient (often linear-time) algo for precisely locating attractors in ReLU-based RNNs. We prove that besides EVGP, bifurcations are a major obstacle in RNN training, but are provably alleviated by training techniques like GTF. (6/6)

020

DurstewitzLab @durstewitzlab.bsky.social · 24/12/2024

In proceedings.mlr.press/v235/goring2... we laid out a general theory for out-of-domain generalization in DSR. We define OODG in DSR as the ability to predict dynamics in unseen dynamical regimes (basins of attraction). We prove that in its most general form, this problem is intractable. (5/6)

140

DurstewitzLab @durstewitzlab.bsky.social · 24/12/2024

Our Almost-Linear RNN openreview.net/pdf?id=sEpSx... shows that simplicity is king, reducing the # of required nonlin. to a bare min., eg learning Lorenz chaos with just 2 ReLUs! The AL-RNN has a direct relation to symbolic dynamics. It strongly facilitates math. analysis of trained models. (4/6)

100

DurstewitzLab @durstewitzlab.bsky.social · 24/12/2024

In multimodal TF we extended this idea to combinations of arbitrary data modalities, illustrating that chaotic attractors can even be learned from just a symbolic encoding, and providing a common dynamical embedding for different modalities: proceedings.mlr.press/v235/brenner... (3/6)

100

DurstewitzLab @durstewitzlab.bsky.social · 24/12/2024

I start with *generalized TF*, which overcomes the explod./vanish. grad. probl. *in training* for any RNN, enabling DSR on highly chaotic and complex real-world data: proceedings.mlr.press/v202/hess23a... Most other DSR work considers only simulated benchm. & struggles with real data. (2/6)

110

DurstewitzLab @durstewitzlab.bsky.social · 24/12/2024

Now may be a good time to introduce our group on bsky with some of our contrib. to dynamical systems reconstruction (DSR) from past year. By DSR we mean learning a *generative surrogate model* of a dyn. process from TS data which reproduces full attractor & generalizes to new init. conditions (1/6)

2234