Sign in

Friedemann Zenke

@fzenke.bsky.social
840 followers 328 following 58 posts

Computational neuroscientist at the FMI. www.zenkelab.org

PostsRepliesMedia
Friedemann Zenke @fzenke.bsky.social · 08/10/2026
8/ Work led by Fabian Mikulasch. 📄 arxiv.org/abs/2609.37789 🌐 info-ldm.github.io Code: github.com/fmi-basel/id...
arxiv.org
Predictive Self-Supervised Learning Provably Identifies Stochastic Signals under Nuisance
Self-supervised learning (SSL) by predicting in latent space, without generating the input data itself, learns highly abstract, useful representations. Intuitively, this success is often attributed to...
051
Friedemann Zenke @fzenke.bsky.social · 08/10/2026
7/ Finally, in ongoing work, the learned latent dynamics can be rolled out from a few observations (t<0) to recover the hopper's signals, including uncertainty estimates.
210
Friedemann Zenke @fzenke.bsky.social · 08/10/2026
6/ Adding a pixel-reconstruction loss to the same model gives latents that partly encode nuisance and no longer allow decoding the state.
110
Friedemann Zenke @fzenke.bsky.social · 08/10/2026
5/ We tested this on a MuJoCo hopper with non-deterministic dynamics and nuisance: random colors, lighting, camera, and noisy backgrounds. The latent predictive model's representation allows to linearly decode the hopper's pose and velocity.
100
Friedemann Zenke @fzenke.bsky.social · 08/10/2026
4/ We show common SSL methods can separate them when two mechanisms work together: predictive MI maximization retains all predictable information; latent distribution matching makes the retained signal identifiable. For Gaussian predictors, we prove recovery to an affine map.
120
Friedemann Zenke @fzenke.bsky.social · 08/10/2026
3/ Prior identifiability theory covers either stochastic signal dynamics (without nuisance) or nuisance (but only deterministic signal dynamics). Video, sensors, and agents acting in the world have to deal with both at once.
100
Friedemann Zenke @fzenke.bsky.social · 08/10/2026
2/ The signals we care about themselves are stochastic. Both the stochastic signals and the nuisance variables make the next observation unpredictable. So how can a predictive model know what to keep and what to ignore?
120
Friedemann Zenke @fzenke.bsky.social · 08/10/2026
1/ Why does predicting in latent space (JEPA, CPC, SimCLR...) work so well on messy data, with changing lighting, camera angles, and busy backgrounds? The usual answer: it can ignore nuisance. But that answer holds a conundrum.
12912
Friedemann Zenke @fzenke.bsky.social · 05/10/2026
9/ Paper: arxiv.org/abs/2610.01373 · Code: github.com/fmi-basel/co... · Project page: ctwm-website.github.io
arxiv.org
Learning Commute-Time-Preserving World Models for Planning
World models allow agents to plan in latent space by choosing a sequence of actions that most reduces the distance to a given goal state. Thus, planning can benefit from latent representations whose d...
030
Friedemann Zenke @fzenke.bsky.social · 05/10/2026
8/ You can see why directly: in PointMaze, CTWM's latent distances track the maze's true commute-time structure far more closely than LeWM's, especially around bottlenecks where it matters most for planning.
130
Friedemann Zenke @fzenke.bsky.social · 05/10/2026
7/ Across six pixel-based goal-reaching tasks (navigation and manipulation) CTWM matches or beats LeWM, a strong task-agnostic baseline, using half the parameters (9M vs. 18M).
140
Friedemann Zenke @fzenke.bsky.social · 05/10/2026
6/ The catch: most self-supervised learning pushes representations to spread out evenly in every direction – which destroys exactly the scaling you need. We combine residual prediction with a log-determinant regularizer instead and prove it recovers the correctly scaled representation.
140
Friedemann Zenke @fzenke.bsky.social · 05/10/2026
5/ Graph theory says yes – if the latent space is aligned with the eigenvectors of the environment's graph Laplacian, and correctly scaled, i.e. choosing the right metric, straight-line distance in that space is commute-time distance.
130
Friedemann Zenke @fzenke.bsky.social · 05/10/2026
4/ Most latent planners pick actions that reduce goal distance in latent space. But is straight-line the right notion of "close"? Commute-time distance measures how long a random walker takes to reach a goal and back. It captures bottlenecks, dead ends, shortcuts. Can latent distance reflect this?
120
Friedemann Zenke @fzenke.bsky.social · 05/10/2026
3/ But a map is only useful if you can measure distances on it. When an AI agent builds its own internal "map" (its latent space) purely from experience, what should distance in that map actually mean?
120
Friedemann Zenke @fzenke.bsky.social · 05/10/2026
2/ Rats explored a maze with no reward in sight. Weeks later, when food finally appeared, they didn't fumble around – instead they beelined straight to it, using shortcuts they never took before. Tolman called this “latent learning”: animals build a map of their world before they know what it's for.
120
Friedemann Zenke @fzenke.bsky.social · 05/10/2026
1/9 New paper led by Michael Hauri & Peter Buttaroni: “Learning Commute-Time-Preserving World Models for Planning.” We show how to shape a world model's latent space so its distances mean something for planning.🧵 📄 arxiv.org/abs/2610.01373 💻 github.com/fmi-basel/co... 🌐 ctwm-website.github.io
1277
Reposted by Friedemann Zenke
Naoshige Uchida @naoshigeuchida.bsky.social · 25/09/2026
Our new work on how belief states could be implemented by attractor-like dynamics in the brain -- an effort started in Starkweather et al. (2017) in collaboration with Sam Gershman, and Jay Hennig, now joined by Scott Linderman! Thank you to all collaborators!
0275
Friedemann Zenke @fzenke.bsky.social · 25/09/2026
Looking forward to the #BernsteinConference next week! Our lab is present with multiple workshop talks and posters. Come, check them out! zenkelab.org/2026/09/the-...
0127
Reposted by Friedemann Zenke
Guillaume Bellec @bellecguill.bsky.social · 23/09/2026
I have a postdoc position open in my NeuroAi lab in Vienna 👌👾🧪🧠 And I will be at Bernstein (workshop only), please reach out if you are interested
1158
Reposted by Friedemann Zenke
Dan Goodman @neural-reckoning.org · 03/09/2026
#SpikingNeuralNetwork people - this year's SNUFA workshop is live! Talks by: ⭐ Eugene Izhikevich ⭐ @giuliadangelo.bsky.social ⭐ Mihai Petrovici ⭐ Susanne Schreiber Submit your abstracts by Sept 25th. Registration is open! More info at: snufa.net/2026/ cc @fzenke.bsky.social @albada.bsky.social
Picture of the SNUFA spiky hedgehog
02416
Reposted by Friedemann Zenke
Tim Kietzmann @timkietzmann.bsky.social · 19/08/2026
📢 We are hiring a full professor in "Intelligence in Biological and Artificial Systems". Come join us at Germany's first and largest cognitive science program. Please share far and wide, and get in touch with any questions. www.uni-osnabrueck.de/en/universit...
uni-osnabrueck.de
129 FB 8 W3 Professorship Intelligence in Biological and Artificial Systems, Institute of Cognitive Science: Uni Osnabrück
05731
Reposted by Friedemann Zenke
Johannes Felsenberg @felsenberg.bsky.social · 31/07/2026
Finally out in @natneuro.nature.com! A forgotten memory isn't always gone. We show that forgotten memories persist as silent memory traces, can be recovered by reminders, and can even be reconstructed into false memories. Excited to finally share this work! www.nature.com/articles/s41...
nature.com
Creating true and false memories from forgotten information in Drosophila - Nature Neuroscience
Forgotten memories in Drosophila persist as silent memory traces and can be recovered by reminder cues. Manipulating reminders can create false memories, showing that memory reconstruction flexibly in...
1214154
Reposted by Friedemann Zenke
Edvard I Moser @edvardmoser.bsky.social · 11/03/2026
Is spatial navigation innate 🧠? Using #NeuroPixels we show that the #torus 🍩 underlying the #GridCell map exists already on day 10 in rats — before pups open eyes and ears and before they start upright walking. 🧵1:4 👇 www.biorxiv.org/content/10.6...
biorxiv.org
318152
Reposted by Friedemann Zenke
Zihan Wu @zihan-wu.bsky.social · 16/06/2026
Can we match self-supervised backpropagation using local learning rules? We show it is possible in our new paper accepted by ICML. We achieve: 1. theoretical equivalence to BP in a controlled setup 2. new SOTA for local learning across image datasets 3. same performance as BP on multiple datasets
3339
Reposted by Friedemann Zenke
Ezekiel Williams @ezekielwilliams.bsky.social · 10/06/2026
1/7 Excited to share my last PhD article, just accepted to ICML 2026! In it, we (me, Alexandre Payeur, Guillaume Lajoie) used dynamical systems theory to study "local" learning in linear recurrent neural networks. See link for the paper, and thread for a brief summary. arxiv.org/abs/2606.00243
arxiv.org
Dynamics and Representation Structure of Local Approximations to Gradient-Based Learning in Linear Recurrent Neural Networks
Biological and neuromorphic recurrent neural networks (RNNs) are subject to spatial and temporal locality constraints on the information that can plausibly be used during learning. A common strategy t...
34916
Friedemann Zenke @fzenke.bsky.social · 03/06/2026
Woot woot!
020
Friedemann Zenke @fzenke.bsky.social · 29/05/2026
Especially when it's your birthday ;-)
010
Friedemann Zenke @fzenke.bsky.social · 07/05/2026
Yes. However, there are several other aspects in DINO that need considering, such as: multi-crop augmentation, centering, and sharpening in the self-distillation loss. Needs thought.
110
Friedemann Zenke @fzenke.bsky.social · 07/05/2026
Work in progress. Still have to figure out how to deal with Dino's categorical head in this framework.
110
Reposted by Friedemann Zenke
Tim Kietzmann @timkietzmann.bsky.social · 06/05/2026
Happy to announce the 3rd iteration of NEAT (Neuro-AI-Talks), which will take place in Osnabrück September 14th-15th 2026. NEAT is a (deliberately small scale) NeuroAI workshop that brings together researchers from neuroscience and AI. www.kietzmannlab.org/neat2026/ More information below 👇
kietzmannlab.org
NEAT 2026
24424
Friedemann Zenke @fzenke.bsky.social · 06/05/2026
7/ Based on these guarantees, we argue that SSL’s success is not primarily based on discarding “irrelevant” information, but instead on constraining representations directly in the latent space.
050
Friedemann Zenke @fzenke.bsky.social · 06/05/2026
6/ Finally, LDM admits identifiability guarantees in the predictive SSL setting on temporal data. Under mild assumptions, learned representations recover the true latent variables up to affine transformations, even with nonlinear predictors.
151
Friedemann Zenke @fzenke.bsky.social · 06/05/2026
5/ We show that the LDM perspective allows us to derive new SSL algorithms. For example, using a Kalman Filter as the predictor enables principled uncertainty quantification of learned nonlinear representations of temporal data.
280
Friedemann Zenke @fzenke.bsky.social · 06/05/2026
4/ Moreover, we revisit the role of mutual information (MI) maximization, which has been shown neither necessary nor sufficient for SSL. We show that, in situations where it appears beneficial, the underlying benefit stems from the implicit LDM effect of practical MI estimators.
160
Friedemann Zenke @fzenke.bsky.social · 06/05/2026
3/ A flurry of existing SSL methods emerge from LDM as special cases under distinct choices of the latent model and entropy estimators. Specifically, we derive SimCLR, VICReg, CPC, BYOL/SimSiam, and JEPA from LDM, providing a unifying account of contrastive, non-contrastive, and stopgrad approaches.
160
Friedemann Zenke @fzenke.bsky.social · 06/05/2026
2/ All objectives derived within the LDM framework consist of (i) an alignment term that maximizes log-likelihood under the assumed latent model and (ii) a uniformity term that maximizes entropy of the embeddings to prevent representational collapse.
150
Friedemann Zenke @fzenke.bsky.social · 06/05/2026
1/7 New paper accepted as ICML spotlight arxiv.org/abs/2605.03517! We unify self-supervised learning (SSL) algorithms (e.g., contrastive, VICReg, stopgrad) via latent distribution matching (LDM), which matches an induced latent distribution to an explicit latent model.
37926
Reposted by Friedemann Zenke
Aaron Milstein @neurosutras.bsky.social · 27/03/2026
Our latest publication grapples with how the brain could implement gradient descent by sending learning targets top-down, gating plasticity with dendritic inhibition, and updating synaptic weights with biologically observed learning rules like BTSP. www.cell.com/cell-reports...
cell.com
Cellular and subcellular specialization enables biology-constrained deep learning
Galloni et al. introduce “dendritic target propagation”: a Dale’s law-compliant learning algorithm for cortical microcircuits with soma- and dendrite-targeting inhibition and realistic connectivity co...
49435
Reposted by Friedemann Zenke
Rebecca Jordan @beckyjordan.bsky.social · 05/03/2026
First preprint from the lab! Using intracellular recordings & analysis of 2-photon imaging data, we show that spiking & neuromodulatory input during experience drive a reorganization of visuomotor inputs in V1 layer 2/3 neurons, consistent with enhanced visuomotor cancellation - bioRxiv link below.
17423
Friedemann Zenke @fzenke.bsky.social · 19/03/2026
Thanks! That's $1M question. Tbh I thought it would be easier to find robust examples. If noise is uncorrelated reconstruction will simply remove it (think denoising autoencoder). I'd put my money on small encoders close to capacity or by adding explicit inductive biases on the latent dynamics.
010
Friedemann Zenke @fzenke.bsky.social · 19/03/2026
3/ Dreamer-CDP uses a JEPA-style predictor over the continuous embeddings instead of pixels. It matches vanilla Dreamer on Crafter and outperforms prior reconstruction-free methods. Thus, reconstruction-free world models are maturing, with potential gains in efficiency & generalization.
040
Friedemann Zenke @fzenke.bsky.social · 19/03/2026
2/ Standard MBRL (e.g. Dreamer) reconstructs images to model the world, potentially wasting capacity on visual details irrelevant to the task. Prior reconstruction-free approaches exist but underperform on benchmarks like Crafter.
110
Friedemann Zenke @fzenke.bsky.social · 19/03/2026
1/3 New paper accepted at ICRL World Model workshop: Dreamer-CDP: Improving Reconstruction-free World Models for RL. We introduce a Dreamer variant that learns world models without reconstructing pixels. arxiv.org/abs/2603.07083
2235
Friedemann Zenke @fzenke.bsky.social · 12/03/2026
Come see our Cosyne 2026 posters! Friday: 2-069 (Atena & Manu), 2-096 (Julian), Saturday: 3-091 (Julia) More info zenkelab.org/2026/03/cosy...
zenkelab.org
Cosyne 2026 – Zenke Lab
0194
Reposted by Friedemann Zenke
FMI science @fmiscience.bsky.social · 13/02/2026
Congrats to Fabian Mikulash, a postdoc in the @fzenke.bsky.social lab, for being awarded a Marie Skłodowska-Curie Actions fellowship! His project aims to develop a new theory—tested with real brain data—explaining how neurons decide when to trust what we see versus what we expect 🧠
081
Reposted by Friedemann Zenke
SueYeon Chung @sueyeonchung.bsky.social · 10/02/2026
Our paper is out in @natneuro.nature.com! www.nature.com/articles/s41... We develop a geometric theory of how neural populations support generalization across many tasks. @zuckermanbrain.bsky.social @flatironinstitute.org @kempnerinstitute.bsky.social 1/14
7278101
Reposted by Friedemann Zenke
Anna Vasilevskaya @loghyr.bsky.social · 30/01/2026
Our work with @georgkeller.bsky.social on testing predictive processing (PP) models in cortex is out on biorvix now! www.biorxiv.org/content/10.6... A short thread on our findings and thoughts on where we should move on from PP below.
biorxiv.org
A functional influence based circuit motif that constrains the set of plausible algorithms of cortical function
There are several plausible algorithms for cortical function that are specific enough to make testable predictions of the interactions between functionally identified cell types. Many of these algorithms are based on some variant of predictive processing. Here we set out to experimentally distinguish between two such predictive processing variants. A central point of variability between them lies in the proposed vertical communication between layer 2/3 and layer 5, which stems from the diverging assumptions about the computational role of layer 5. One assumes a hierarchically organized architecture and proposes that, within a given node of the network, layer 5 conveys unexplained bottom-up input to prediction error neurons of layer 2/3. The other proposes a non-hierarchical architecture in which internal representation neurons of layer 5 provide predictions for the local prediction error neurons of layer 2/3. We show that the functional influence of layer 2/3 cell types on layer 5 is incompatible with the hierarchical variant, while the functional influence of layer 5 cell types on prediction error neurons of layer 2/3 is incompatible with the non-hierarchical variant. Given these data, we can constrain the space of plausible algorithms of cortical function. We propose a model for cortical function based on a combination of a joint embedding predictive architecture (JEPA) and predictive processing that makes experimentally testable predictions. ### Competing Interest Statement The authors have declared no competing interest. Swiss National Science Foundation, https://ror.org/00yjd3n13 Novartis Foundation, https://ror.org/04f9t1x17 European Research Council, https://ror.org/0472cxd90, 865617
25016
Reposted by Friedemann Zenke
Edvard I Moser @edvardmoser.bsky.social · 28/01/2026
The hippocampal map has its own attentional control signal! Our new study reveals that theta #sweeps can be instantly biased towards behaviourally relevant locations. See 📹 in post 4/6 and preprint here 👉 www.biorxiv.org/content/10.6... 🧵(1/6)
biorxiv.org
Attention-like regulation of theta sweeps in the brain's spatial navigation circuit
Spatial attention supports navigation by prioritizing information from selected locations. A candidate neural mechanism is provided by theta-paced sweeps in grid- and place-cell population activity, which sample nearby space in a left-right-alternating pattern coordinated by parasubicular direction signals. During exploration, this alternation promotes uniform spatial coverage, but whether sweeps can be flexibly tuned to locations of particular interest remains unclear. Using large-scale Neuropixels recordings in freely-behaving rats, we show that sweeps and direction signals are rapidly and dynamically modulated: they track moving targets during pursuit, precede orienting responses during immobility, and reverse during backward locomotion — without prior spatial learning. Similar modulation occurs during REM sleep. Canonical head-direction signals remain head-aligned. These findings identify sweeps as a flexible, attention-like mechanism for selectively sampling allocentric cognitive maps. ### Competing Interest Statement The authors have declared no competing interest. European Research Council, Synergy Grant 951319 (EIM) The Research Council of Norway, Centre of Neural Computation 223262 (EIM, MBM), Centre for Algorithms in the Cortex 332640 (EIM, MBM), National Infrastructure grant (NORBRAIN, 295721 and 350201) The Kavli Foundation, https://ror.org/00kztt736 Ministry of Science and Education, Norway (EIM, MBM) Faculty of Medicine and Health Sciences; NTNU, Norway (AZV)
418362
Reposted by Friedemann Zenke
Sam Gershman @gershbrain.bsky.social · 09/01/2026
With some trepidation, I'm putting this out into the world: gershmanlab.com/textbook.html It's a textbook called Computational Foundations of Cognitive Neuroscience, which I wrote for my class. My hope is that this will be a living document, continuously improved as I get feedback.
16590237