Sign in

Gaia Molinaro

@gaiamolinaro.bsky.social
614 followers 270 following 32 posts

How do we learn to achieve our goals? 🧠 Microsoft Research Scientist 💻 UC Berkeley PhD 🎓

PostsRepliesMedia
Gaia Molinaro @gaiamolinaro.bsky.social · 08/10/2026
Our reward function compression work is now out in Nature Communications! 🎉 In it, we present and empirically test a computational framework that formalizes the trade-offs between goal flexibility and learning efficiency. @annecollins.bsky.social @natureportfolio.nature.com doi.org/10.1038/s414...
Conceptual model of goal-directed reinforcement learning with reward
function compression. In our proposed conceptual model, we break down goald-dependent learning into several parts. Combined with other learning components, an RL learning algorithm informs value estimates W on a given trial t, which are entered into a policy. The agent selects an action a through a joint policy, to which the environment responds by outputting outcome information o and a new state s. A working memory (WM) component organizes and maintains task-relevant information (here, broadly intended and dubbed the task information), used to support the reinforcement learning (RL) process. a Initially, value attribution to o is operated by WM,which maintains the current goal (g) and supports its matching to the experienced outcome to generate a reward r used by the RL agent. WM also holds a trace of recently experienced goals. b If goals share enough similar features,
a compressed reward function can be created, which replaces active goal maintenance and can, over time, be transferred to long-term memory storage. Eventually, the compressed reward function can be directly employed to assign value to experienced outcomes, sparing resources for other WM tasks.
08927
Gaia Molinaro @gaiamolinaro.bsky.social · 03/10/2026
Well deserved, complimenti Francesco!
011
Gaia Molinaro @gaiamolinaro.bsky.social · 30/09/2026
Huge congrats!!!! I can't wait to see what fantastic research comes out of your newly minted lab!
010
Gaia Molinaro @gaiamolinaro.bsky.social · 19/09/2026
These innovations also leave us with several open questions about the characteristics of human goals, AI goals, and synergistic self-driven ("autotelic") systems. Answering them, we believe, will require reciprocal exchange between AI and cognitive science. www.sciencedirect.com/science/arti...
sciencedirect.com
021
Gaia Molinaro @gaiamolinaro.bsky.social · 19/09/2026
We explain how, by operating over formats such as natural language and code, foundation models allow agents to: ✨ Generate more abstract and compositional goals. ✨ Approximate human judgments of interestingness and learnability. ✨ Support richer loops of problem generation and problem solving.
Illustration of the evolution of goal-directed learning agents across three paradigms: standard RL, pre-foundation model autotelic loop, foundation model-based autotelic loop.
140
Gaia Molinaro @gaiamolinaro.bsky.social · 19/09/2026
The ability to pursue self-determined goals is key to human intelligence. Could foundation models unlock similar capabilities in artificial agents? We explore this possibility in a short review — fresh off the press! @ccolas.bsky.social @pyoudeyer.bsky.social @annecollins.bsky.social
1143
Gaia Molinaro @gaiamolinaro.bsky.social · 28/08/2026
This was a fun collaboration led by Jérémy Perez and @ccolas.bsky.social, with @pyoudeyer.bsky.social, @maximederex.bsky.social, and @clemmoulinfrier.bsky.social
030
Gaia Molinaro @gaiamolinaro.bsky.social · 28/08/2026
Can you think of a goal that’s shaped by your culture? Or one that changed society as a result of being pursued? If not, check out the paper for some hints! 👀 onlinelibrary.wiley.com/doi/10.1111/...
onlinelibrary.wiley.com
The Cultural Evolution of Human Goals
Characterizing the reciprocal interactions between cultural environments and individual goals is crucial for building a comprehensive account of human cultural evolution. This paper introduces the cu...
160
Gaia Molinaro @gaiamolinaro.bsky.social · 28/08/2026
To tackle these ideas, we propose framing individuals as “cultural autotelic agents” and outline ways to study how culture shapes the goals people generate, anticipate, and select.
110
Gaia Molinaro @gaiamolinaro.bsky.social · 28/08/2026
Curiosity and intrinsic motivation help keep this cycle open-ended, pushing people toward diverse and increasingly complex pursuits even when there is no obvious external reward.
120
Gaia Molinaro @gaiamolinaro.bsky.social · 28/08/2026
Why do some goals spread across people and generations, while others emerge only transiently? We argue that human goals are both drivers and products of cultural evolution: our individual pursuits are shaped by our cultural environments, and the goals we pursue in turn reshape those environments.
1189
Reposted by Gaia Molinaro
Aly Lidayan @aliday.bsky.social · 04/08/2026
New paper! Setting goals is so useful in complex settings because it can reduce the environment's complexity to only the goal-relevant features. But for humans, reframing comes at a cost. Could this explain why we're so bad at abandoning goals that are no longer worthwhile? 🧵
1133
Reposted by Gaia Molinaro
Jonathan Tsay @tsay.bsky.social · 24/04/2026
Dr. Matthew Warburton and I put together a guide on "how to conduct behavioural experiments online"—something we wish we had when we were getting started. Hopefully it’s useful, and we’d very much welcome any feedback!
13114
Gaia Molinaro @gaiamolinaro.bsky.social · 13/02/2026
What makes new goals difficult to reach? As I discussed with @alisonpstevens.bsky.social for @sciencenews.bsky.social, the issue might be holding precise but costly memories of the target. As we internalize rules for what success looks like, goals become more attainable. @annecollins.bsky.social
snexplores.org
Big goals can feel overwhelming. Here’s how to stay on track
Aiming for something big? Break that big goal into smaller, manageable steps. Then narrow your focus. This will keep your eyes on the prize.
063
Reposted by Gaia Molinaro
annecollins.bsky.social @annecollins.bsky.social · 17/11/2025
My paper is out! Computational modeling of error patterns during reward-based learning show evidence that habit learning (value free!) supplements working memory in 7 human data sets. rdcu.be/eQjLN
rdcu.be
A habit and working memory model as an alternative account of human reward-based learning
Nature Human Behaviour - In this study, Collins proposes an alternative dual-process (working memory and habit) model of reinforcement learning in humans.
213349
Gaia Molinaro @gaiamolinaro.bsky.social · 11/11/2025
Next week, I'll be presenting my work with @annecollins.bsky.social @collinsccnlab.bsky.social on reward function compression in human RL at @sfn.org. Join me at the "Adaptive Choice" nanosymposium on Nov 16th or dm me if you'd like to meet up!
0112
Gaia Molinaro @gaiamolinaro.bsky.social · 09/09/2025
That's a fascinating argument, thanks for bringing it to my attention, Jim!
000
Gaia Molinaro @gaiamolinaro.bsky.social · 09/09/2025
📢 New preprint! How do humans learn from arbitrary, abstract goals? We show that, when goal spaces can be compressed, costly working-memory processes give way to internalized reward functions, enabling efficient goal-dependent reinforcement learning. @annecollins.bsky.social arxiv.org/abs/2509.06810
arxiv.org
Reward function compression facilitates goal-dependent reinforcement learning
Reinforcement learning agents learn from rewards, but humans can uniquely assign value to novel, abstract outcomes in a goal-dependent manner. However, this flexibility is cognitively costly, making l...
25924
Reposted by Gaia Molinaro
Pierre-Yves Oudeyer @pyoudeyer.bsky.social · 25/04/2025
Generative AI is a cultural transmission technology: it plays a growing role in generation, selection and transmission of ideas/opinions in human society 🧠🔄🌐 And yet we understand very little of this dynamics at this point 🤔❓ A step forward is our #ICLR2025 paper !👇
171
Gaia Molinaro @gaiamolinaro.bsky.social · 23/04/2025
Work led by Jérémy Perez with Corentin Leger @kovacgrgur.bsky.social Cédric Colas @clemmoulinfrier.bsky.social @pyoudeyer.bsky.social @maximederex.bsky.social
020
Gaia Molinaro @gaiamolinaro.bsky.social · 23/04/2025
Find out more here arxiv.org/abs/2407.04503 or check out our companion website sites.google.com/view/telepho...
arxiv.org
When LLMs Play the Telephone Game: Cumulative Changes and Attractors in Iterated Cultural Transmissions
As large language models (LLMs) start interacting with each other and generating an increasing amount of text online, it becomes crucial to better understand how information is transformed as it passe...
130
Gaia Molinaro @gaiamolinaro.bsky.social · 23/04/2025
What's wrong with evaluating #LLMs after a single interaction? Come find out @iclr-conf.bsky.social and learn how cultural attraction theory can help us do better. Poster #288, 10 am.
172
Gaia Molinaro @gaiamolinaro.bsky.social · 12/03/2025
This paper concludes a long, exciting journey, which started several years ago when I first met Moshe. It's been a fantastic ride! Thank you Moshe, Bar Ilan University, and Onward Israel for enabling it ♥️ and the editors Deniz Vatansever and Jessica Andrews-Hanna for inviting us to contribute!
050
Gaia Molinaro @gaiamolinaro.bsky.social · 12/03/2025
In light of its similar merits, ST deserves at least the same care, protection, and appreciation we devote to children’s play. Get access to the full article here: www.sciencedirect.com/science/arti... @elsevierconnect.bsky.social
sciencedirect.com
Spontaneous thought as play: the value of fictional goals in the default mode network
Given its prevalence in our wakeful mental activity, spontaneous thought (ST) has been attributed several roles in cognition, most of which engage the…
182
Gaia Molinaro @gaiamolinaro.bsky.social · 12/03/2025
Similarly, we propose that ST provides a cognitive platform for simulating outcomes and ideas that extend beyond immediate constraints. As long as a fictional goal provides enough structure, even unrealistic STs can contribute to our cognitive flexibility.
150
Gaia Molinaro @gaiamolinaro.bsky.social · 12/03/2025
In play, children engage in "fictional" goals. @junyi.bsky.social, @joshtenenbaum.bsky.social and Laura Schulz argued that these seemingly trivial activities enhance children's capacity to plan, innovate, and simulate complex scenarios.
140
Gaia Molinaro @gaiamolinaro.bsky.social · 12/03/2025
To answer this question, we draw an analogy between ST and children's play.
140
Gaia Molinaro @gaiamolinaro.bsky.social · 12/03/2025
The usefulness of ST is often connected to the proactive simulation of future scenarios related to our goals. But what about ST that engages in frivolous or highly unlikely situations?
130
Gaia Molinaro @gaiamolinaro.bsky.social · 12/03/2025
Spontaneous thought (ST, the relatively unrestricted thought process that accompanies much of our waking life) benefits cognition in several ways, from memory consolidation to mood enhancement. Check out work by Judith Mildner and @dianatamir.bsky.social for an awesome recent perspective on this.
140
Gaia Molinaro @gaiamolinaro.bsky.social · 12/03/2025
Have you ever gotten lost in fantasies and thought to yourself, "What a waste of time"? Well, think again! In our new piece in Current Opinion in Behavioral Sciences, Moshe Bar and I explain one reason why spontaneous thoughts, even when seemingly useless, may be key to human intelligence.
1407
Gaia Molinaro @gaiamolinaro.bsky.social · 11/12/2024
Stop by @neuripsconf.bsky.social poster #3909 11 am - 2 pm today if you want to wish me a happy birthday, hear about goal selection in RL, or both! @pyoudeyer.bsky.social @annecollins.bsky.social
141
Gaia Molinaro @gaiamolinaro.bsky.social · 18/11/2024
I'm excited to announce that this work has been accepted at @blog.neurips.cc.web.brid.gy 🧠🤖 We hope to spark conversations on goal selection in biological and artificial agents. Check it out at openreview.net/forum?id=Gbq... With Cédric Colas, Pierre-Yves Oudeyer, & Anne Collins
openreview.net
Latent Learning Progress Drives Autonomous Goal Selection in Human...
Humans are autotelic agents who learn by setting and pursuing their own goals. However, the precise mechanisms guiding human goal selection remain unclear. Learning progress, typically measured as...
1156
Gaia Molinaro @gaiamolinaro.bsky.social · 17/06/2024
No time to read? A 5-minute summary of this paper is now out on YouTube! youtu.be/fkPt9a5OvTs?...
youtu.be
Latent learning progress guides hierarchical goal selection in humans - Gaia Molinaro (UC Berkeley)
Spotlight talk at the 5th International Convention on the Mathematics of Neuroscience and Artificial Intelligence, Rome, 2024 (https://neuromonster.org). Recorded and hosted with generous funding from the Kavli Foundation, Gatsby Foundation, Templeton Foundation, Harvard University, European Research Council, Artificial Intelligence Journal, and Google DeepMind. © Thinking About Thinking, Inc, a 501(c)3 Nonprofit registered in New Jersey, USA. https://thinkingaboutthinking.org
010
Gaia Molinaro @gaiamolinaro.bsky.social · 10/06/2024
Thanks for flagging @ebonawitz.bsky.social! Here it is osf.io/preprints/ps...
osf.io
OSF
010
Gaia Molinaro @gaiamolinaro.bsky.social · 08/06/2024
How do we decide which goals to pursue? Here, we introduce the notion of *latent learning progress* to explain human goal selection and inspire advances in the development of autotelic machines. osf.io/preprints/ps... With Cédric Colas, Pierre-Yves Oudeyer, and Anne Collins
osf.io
OSF
273
Gaia Molinaro @gaiamolinaro.bsky.social · 14/11/2023
The notion that goals are central to human cognition is intuitive, yet learning and decision-making researchers have often overlooked the topic. In this Trends in Cognitive Sciences review, @annecollins.bsky.social and I propose it’s time to start studying goals in their own right 🧠 t.co/He6PIpQdt9
An illustration showing a soccer ball shaping a net of neurons, trying to convey the idea that "goals" shape cognition
14523