Since I wrote my last post about AI and math, the theorems have continued to fall.

When I was around 11 I heard for the first time about Fermat’s Last Theorem. I was immediately captivated by the problem statement, as well as by the accompanying story, and made a fairly ser…

Some reflections on connectionism and the basis of higher-level cognition

Flexible decision making depends on retrieving and recombining memories. Yet because this process unfolds covertly, its governing principles remain unknown. Here we use gaze reinstatement to uncover t...

When do machine learning systems fail to generalize, and what mechanisms could improve their generalization? Here, we draw inspiration from cognitive science to argue that one weakness of...

The success of large language models (LLMs) across many domains of AI research has generated intense debate. Some attribute their impressive performance on complex tasks to human-like linguistic and ...

Reflections on catastrophic interference, plasticity, and learning for the future in the era of large language models

How can cognitive science build generalizable theories that span the full scope of natural situations and behaviors? We argue that progress in Artificial Intelligence (AI) offers timely opportunities ...

A recent paper (van Rooij et al. 2024) claims to have proved that achieving human-like intelligence using learning from data is intractable in a complexity-theoretic sense. We point out that the proof...

Language Models (LMs) exhibit two distinct mechanisms for knowledge acquisition: in-weights learning (i.e., encoding information within the model weights) and in-context learning (ICL). Although these...

A central approach in neuroscience is to analyze neural representations as a means to understand a system's function, through the use of methods like principal component analysis, regression, and repr...

Today I want to write a bit about the philosophy I think underlies much of the work that my collaborators and I (as well as many other researchers that I respect) have done on understanding artificial...

Dileep George is joining Astera as Head of AI, leading our AGI research division. Working alongside our Chief Scientist Doris Tsao, he and the team will explore novel, brain-inspired computational arc...

Or: how I learned to stop worrying and love the memorization

Using artificial neural networks applied to human data, Eckstein et al. show that good models of reinforcement learning require memory components that track representations of the past.

The epilogue to a series on Cognitive Science and AI

Nicholas and Mattar found that people use episodic memory to make decisions when it is unclear what will be needed in the future. These findings reveal how the rich representational capacity of episod...

Nature Human Behaviour - Language is often thought to be represented through hierarchically structured units. Nielsen and Christiansen find that non-hierarchical structures are present across...

A proof is only as good as its assumptions

#3 in a series on cognitive science and AI

#2 in a series on cognitive science and AI

First post in a series on cognitive science and AI