iiinova — where your digital self keeps evolving

Reece Keller

@reecedkeller.bsky.social

101 followers 7 following 10 posts

CS+Neuro @cmu.edu‬ PhD Student with Xaq Pitkow and @anayebi.bsky.social‬ working on autonomous embodied AI.

Posts Replies Media

Reece Keller @reecedkeller.bsky.social · 05/06/2025

10/ Animal-like autonomy—flexibly adapting to new environments without supervision—is a key ingredient of general intelligence. Our work shows this hinges on 1) a predictive world model and 2) memory primitives that ground these predictions in ethologically relevant contexts.

3202

Reece Keller @reecedkeller.bsky.social · 05/06/2025

9/ Finally, we show that the neural-glial circuit proposed in Mu et al. (2019) emerges from the latent dynamics of 3M-Progress agents. Thanks to my collaborators Alyn Kirsch and Felix Pei, and to Xaq Pitkow for his continued support! Paper link: arxiv.org/abs/2506.00138

1170

Reece Keller @reecedkeller.bsky.social · 05/06/2025

8/ 3M-Progress agents achieve the best alignment with brain data compared to existing intrinsic drives and data-driven controls. Together with the behavioral alignment, 3M-Progress agents saturate inter-animal consistency and thus pass the NeuroAI Turing test on this dataset.

1120

Reece Keller @reecedkeller.bsky.social · 05/06/2025

7/ 3M-Progress agents exhibit stable transitions between active and passive states that closely match real zebrafish behavior.

1140

Reece Keller @reecedkeller.bsky.social · 05/06/2025

6/ The agent learns a forward dynamics model and measures the divergence between this model and a frozen ethological memory. This model-memory-mismatch (3M) is tracked over time (w/ gamma-progress) to form the final intrinsic reward.

1150

Reece Keller @reecedkeller.bsky.social · 05/06/2025

5/ First, we construct two environments extending the dm-control suite: one that captures the basic physics of zebrafish ecology (reactive fluid forces and drifting currents), and one that replicates the head-fixed experimental protocol in Yu Mu and @mishaahrens.bsky.social et al. 2019.

1161

Reece Keller @reecedkeller.bsky.social · 05/06/2025

4/ To bridge this gap, we introduce 3M-Progress, which reinforces behavior that systematically aligns with an ethological memory. 3M-Progress agents capture nearly all the variability in behavioral and whole-brain calcium recordings in autonomously behaving larval zebrafish.

2180

Reece Keller @reecedkeller.bsky.social · 05/06/2025

3/ Existing model-based intrinsic motivation algorithms (e.g. learning progress, prediction-error) exhibit non-stationary and saturating reward dynamics, leading to transient behavioral strategies that fail to capture the robust nature of ethological animal behavior.

2180

Reece Keller @reecedkeller.bsky.social · 05/06/2025

2/ Model-based intrinsic motivation is a class of exploration methods in RL that leverage predictive world models to generate an intrinsic reward signal. This signal is completely self-supervised and can guide behavior in sparse-reward or reward-free environments.

1240

Reece Keller @reecedkeller.bsky.social · 05/06/2025

1/ I'm excited to share recent results from my first collaboration with the amazing @anayebi.bsky.social and @leokoz8.bsky.social ! We show how autonomous behavior and whole-brain dynamics emerge in embodied agents with intrinsic motivation driven by world models.

212527