Sign in

Simon Schug

@smonsays.bsky.social
988 followers 244 following 28 posts

postdoc @princeton computational cognitive science ∪ machine learning smn.one

PostsRepliesMedia
Simon Schug @smonsays.bsky.social · 17/09/2026
Is that an issue? We think yes: How can we robustly establish cognitive abilities in reasoning models if those abilities are so closely tied to the particular contexts they were evaluated in? You can check your favorite model's systematicity using our code: github.com/smonsays/sys...
github.com
GitHub - smonsays/systematicity-eval: Offical code for the paper ''Thought without systematicity? Evaluating reasoning models on rule induction tasks"
Offical code for the paper ''Thought without systematicity? Evaluating reasoning models on rule induction tasks" - smonsays/systematicity-eval
060
Simon Schug @smonsays.bsky.social · 17/09/2026
While reasoning models can be systematic, we find that many model behaviors lack systematicity.
Systematicity of reasoning models on rule induction tasks. We evaluate reasoning models on compositional families of rule induction tasks. For each particular task within a family, we generate K=5 structurally equivalent task variants per type of task invariance. For each set of task variations, we report whether any of the variations was solved correctly, the fraction of variations solved correctly and whether all variations were solved correctly and report the average over task variation sets within a task family. We find that almost all models are perfectly systematic on the Boolean category learning task family, but see a notable systematicity gap between the ability to solve any task variation and solving all task variations on the remaining task families.
121
Simon Schug @smonsays.bsky.social · 17/09/2026
We then independently evaluate the same model across such task variations and verify whether it consistently solves all variations if it solved any.
Bar plot visualization of our systematicity metrics.
120
Simon Schug @smonsays.bsky.social · 17/09/2026
We use compositional structure in four rule induction task families to create tasks with structurally equivalent variations.
Evaluating systematicity in rule induction tasks with compositional structure. (A) Compositional task families allow to construct large numbers of tasks with related structure. The shown task is a simplified version of the grammar-based instruction-learning task presented in Section 4.1. (B) Each rule induction task requires inferring hidden rules on a set of input -> output support examples to solve novel query examples. (C) The compositional task structure allows creating systematic variations of structurally equivalent tasks. (D) Reasoning models can be independently evaluated across structurally equivalent task variants to assess their systematicity.
120
Simon Schug @smonsays.bsky.social · 17/09/2026
Systematicity of thought is the idea that being able to understand one thing implies that one also understands certain variations of that thing. These variations are often the result of compositional structure.
120
Simon Schug @smonsays.bsky.social · 17/09/2026
Human thought is thought to be systematic: Do reasoning models have systematicity of thought? With @brendenlake.bsky.social we study this question in our new preprint arxiv.org/abs/2609.13948 Thread 🧵
14014
Reposted by Simon Schug
Tom McCoy @rtommccoy.bsky.social · 20/08/2026
Since many are starting grad school soon, let me re-share my One Big Tip™️ for research! Research involves many skills - collaborating, writing, presenting, etc. But many of these skills can be unified under a single overarching ability: theory of mind Blog post link in reply
Illustration of the blog post's main argument, summarized as: "Theory of Mind as a Central Skill for Researchers: Research involves many skills.If each skill is viewed separately, each one takes a long time to learn. These skills can instead be connected via theory of mind – the ability to reason about the mental states of others. This allows you to transfer your abilities across areas, making it easier to gain new skills."
28019
Reposted by Simon Schug
Naomi Saphra @nsaphra.bsky.social · 09/08/2026
I've been unsettled lately when reading messages and papers. It feels like I'm dissociating. Everything seems a bit alien, even if it's completely human. I've had a realization: When our simulations finally exited the Uncanny Valley, they brought the Uncanny with them.
nsaphra.net
Life on the Uncanny Precipice | Naomi Saphra
We were wrong about the Uncanny Valley.
1026561
Simon Schug @smonsays.bsky.social · 09/04/2026
LLM agents are a serious problem for online experiments. It is very easy to use them and very hard to spot them. What can researchers do? With @brendenlake.bsky.social, we suggest detecting LLMs based on their lack of human cognitive constraints in our #CogSci2026 paper: arxiv.org/abs/2604.00016
0254
Reposted by Simon Schug
Minas Karamanis @minaskar.bsky.social · 30/03/2026
Hey, I wrote a thing about AI in astrophysics ergosphere.blog/posts/the-ma...
ergosphere.blog
The machines are fine. I'm worried about us.
On AI agents, grunt work, and the part of science that isn't replaceable.
1101781531
Reposted by Simon Schug
Mark Histed @markhisted.org · 29/03/2026
The hardest part of science is posing the right question, not answering it.
424052
Reposted by Simon Schug
Kai Sandbrink @ackaisa.bsky.social · 26/03/2026
Excited that my paper on metacontrol in humans and neural networks with @summerfieldlab.bsky.social and @lhuntneuro.bsky.social is out in PNAS! We examine the way that predictive representations of control enable behavioral adaptation across settings, and pathologies: www.pnas.org/doi/10.1073/...
04116
Reposted by Simon Schug
Sam Gershman @gershbrain.bsky.social · 14/03/2026
I think Fodor & Pylyshyn's 1988 paper is possibly the most mischaracterized paper in the history of cognitive science. It's often cited as arguing that neural networks cannot achieve systematicity, compositionality, and productivity. But that's not what they actually argue...
29221
Reposted by Simon Schug
Jonathan Nicholas @jonathannicholas.bsky.social · 23/01/2026
Our experiences have countless details, and it can be hard to know which matter. How can we behave effectively in the future when, right now, we don't know what we'll need? Out today in @nathumbehav.nature.com , @marcelomattar.bsky.social and I find that people solve this by using episodic memory.
nature.com
Episodic memory facilitates flexible decision-making via access to detailed events - Nature Human Behaviour
Nicholas and Mattar found that people use episodic memory to make decisions when it is unclear what will be needed in the future. These findings reveal how the rich representational capacity of episod...
713049
Reposted by Simon Schug
C. Thi Nguyen @add-hawk.bsky.social · 22/01/2026
Last term I tried an experiment: I walked into my Tech and Design Ethics class, admitted that I had *no idea* what to do about ChatGPT - so I would let them figure it out. As in: their first project was to decide and write the ChatGPT policy for the class. Here's what happened:
242369864
Reposted by Simon Schug
Griffiths Computational Cognitive Science Lab @cocoscilab.bsky.social · 18/12/2025
Excited to announce a new book telling the story of mathematical approaches to studying the mind, from the origins of cognitive science to modern AI! The Laws of Thought will be published in February and is available for pre-order now.
217041
Reposted by Simon Schug
Erin Grant @eringrant.me · 06/12/2025
Thrilled to start 2026 as faculty in Psych & CS @ualberta.bsky.social + Amii.ca Fellow! 🥳 Recruiting students to develop theories of cognition in natural & artificial systems 🤖💭🧠. Find me at #NeurIPS2025 workshops (speaking coginterp.github.io/neurips2025 & organising @dataonbrainmind.bsky.social)
410928
Reposted by Simon Schug
Paul Masset @paulmasset.bsky.social · 19/11/2025
I am recruiting graduate students for the experimental side of my lab @mcgill.ca for admission in Fall 2026! Get in touch if you're interested in how brain circuits implement distributed computation, including dopamine-based distributed RL and probabilistic representations.
13730
Reposted by Simon Schug
Brenden Lake @brendenlake.bsky.social · 03/11/2025
Checking out the Princeton trails on our lab retreat
0281
Simon Schug @smonsays.bsky.social · 04/11/2025
📄 Paper: arxiv.org/abs/2507.07207 💻 Code: github.com/smonsays/sca...
030
Simon Schug @smonsays.bsky.social · 04/11/2025
But, not all training distributions enable compositional generalization -- even with scale. Strategically choosing the training data matters a lot.
120
Simon Schug @smonsays.bsky.social · 04/11/2025
We prove that MLPs can implement a general class of compositional tasks ("hyperteachers") using only a linear number of neurons in the number of modules, beating the exponential!
110
Simon Schug @smonsays.bsky.social · 04/11/2025
It turns out that simply scaling multilayer perceptrons / transformers can lead to compositional generalization.
110
Simon Schug @smonsays.bsky.social · 04/11/2025
Most natural data has compositional structure. This leads to a combinatorial explosion that is impossible to fully cover in the training data. It might be tempting to think that we need to equip neural network architectures with stronger symbolic priors to capture this compositionality, but do we?
110
Simon Schug @smonsays.bsky.social · 04/11/2025
Does scaling lead to compositional generaliztation? Our #NeurIPS2025 Spotlight paper suggests that it can -- with the right training distribution. 🧵 A short thread:
Plots showing how scaling model size and data size leads to compositional generalizationA generated image composition of a clock inside a treasure chest inside a transparent cube.
1151
Reposted by Simon Schug
Brenden Lake @brendenlake.bsky.social · 12/06/2025
I'm joining Princeton University as an Associate Professor of Computer Science and Psychology this fall! Princeton is ambitiously investing in AI and Natural & Artificial Minds, and I'm excited for my lab to contribute. Recruiting postdocs and Ph.D. students in CS and Psychology — join us!
Nassau Hall. Photo credit to Debbie and John O'Boyle
3472
Simon Schug @smonsays.bsky.social · 25/04/2025
Are transformers smarter than you? Hypernetworks might explain why. Come checkout our Oral at #ICLR tomorrow (Apr 26th, poster at 10:00, Oral session 6C in the afternoon). openreview.net/forum?id=V4K...
1100
Reposted by Simon Schug
Taylor Webb @taylorwwebb.bsky.social · 10/03/2025
LLMs have shown impressive performance in some reasoning tasks, but what internal mechanisms do they use to solve these tasks? In a new preprint, we find evidence that abstract reasoning in LLMs depends on an emergent form of symbol processing arxiv.org/abs/2502.20332 (1/N)
arxiv.org
Emergent Symbolic Mechanisms Support Abstract Reasoning in Large Language Models
Many recent studies have found evidence for emergent reasoning capabilities in large language models, but debate persists concerning the robustness of these capabilities, and the extent to which they ...
511835
Reposted by Simon Schug
Konrad Kording @kordinglab.bsky.social · 14/02/2025
New blog post: The principle of neuroscience. medium.com/@kording/the...
medium.com
The Principle of Neural Science
I first encountered Principles of Neural Science as a young student of neuroscience. The book was filled with delightful narratives…
3465
Reposted by Simon Schug
Marine Schimel @marineschimel.bsky.social · 31/01/2025
For my first Bluesky post, I'm very excited to share a thread on our recent work with Mitra Javadzadeh, investigating how connections between cortical areas shape computations in the neocortex! [1/7] www.biorxiv.org/content/10.1...
biorxiv.org
Dynamic consensus-building between neocortical areas via long-range connections
The neocortex is organized into functionally specialized areas. While the functions and underlying neural circuitry of individual neocortical areas are well studied, it is unclear how these regions op...
12011
Reposted by Simon Schug
Guillaume Bellec @bellecguill.bsky.social · 08/01/2025
Pre-print 🧠🧪 Is mechanism modeling dead in the AI era? ML models trained to predict neural activity fail to generalize to unseen opto perturbations. But mechanism modeling can solve that. We say "perturbation testing" is the right way to evaluate mechanisms in data-constrained models 1/8
412747
Reposted by Simon Schug
Mark D Humphries @markdhumphries.bsky.social · 30/12/2024
Cutting it a bit fine, but here’s my review of the year in neuroscience for 2024 The eighth of these, would you believe? We’ve got dark neurons, tiny monkeys, the most complete brain wiring diagram ever constructed, and much more… Published on The Spike Enjoy! medium.com/the-spike/20...
medium.com
2024: A Review of the Year in Neuroscience
Feeling a bit wired
719171
Reposted by Simon Schug
Kris Jensen @kristorpjensen.bsky.social · 21/12/2024
I wrote an introduction to RL for neuroscience last year that was just published in NBDT: tinyurl.com/5f58zdy3 This review aims to provide some intuition for and derivations of RL methods commonly used in systems neuroscience, ranging from TD learning through the SR to deep and distributional RL!
tinyurl.com
An introduction to reinforcement learning for neuroscience | Published in Neurons, Behavior, Data analysis, and Theory
By Kristopher T. Jensen. Reinforcement learning for neuroscientists
613033
Reposted by Simon Schug
Ben Recht @beenwrekt.bsky.social · 20/12/2024
Stitching component models into system models has proven difficult in biology. But how much easier has it been in engineering? www.argmin.net/p/monster-mo...
argmin.net
Monster Models
Systems-level biology is hard because systems-level engineering is hard.
3122
Reposted by Simon Schug
Badr AlKhamissi @bkhmsi.bsky.social · 19/12/2024
🚨 New Paper! Can neuroscience localizers uncover brain-like functional specializations in LLMs? 🧠🤖 Yes! We analyzed 18 LLMs and found units mirroring the brain's language, theory of mind, and multiple demand networks! w/ @gretatuckute.bsky.social, @abosselut.bsky.social, @mschrimpf.bsky.social 🧵👇
210325
Reposted by Simon Schug
Blake Richards @tyrellturing.bsky.social · 16/12/2024
1/ Okay, one thing that has been revealed to me from the replies to this is that many people don't know (or refuse to recognize) the following fact: The unts in ANN are actually not a terrible approximation of how real neurons work! A tiny 🧵. 🧠📈 #NeuroAI #MLSky
2115238
Reposted by Simon Schug
Razvan Pascanu @razvan-pascanu.bsky.social · 15/12/2024
For my first post on Bluesky .. I'll start by announcing our 2025 edition of EEML which will be in Sarajevo :) ! I'm really excited about it and hope to see many of you there. Please follow the website (and Bluesky account) for more details which are coming soon ..
1337
Reposted by Simon Schug
Markus Meister @mameister4.bsky.social · 14/11/2024
Have you had private doubts whether we'll ever understand the brain? Whether we'll be able explain psychological phenomena in an exhaustive way that ranges from molecules to membranes to synapses to cells to cell types to circuits to computation to perception and behavior?
13911
Reposted by Simon Schug
Andrew Lampinen @lampinen.bsky.social · 10/12/2024
What counts as in-context learning (ICL)? Typically, you might think of it as learning a task from a few examples. However, we’ve just written a perspective (arxiv.org/abs/2412.03782) suggesting interpreting a much broader spectrum of behaviors as ICL! Quick summary thread: 1/7
arxiv.org
The broader spectrum of in-context learning
The ability of language models to learn a task from a few examples in context has generated substantial interest. Here, we provide a perspective that situates this type of supervised few-shot learning...
212232
Reposted by Simon Schug
Kai Sandbrink @ackaisa.bsky.social · 03/12/2024
Thrilled to share our NeurIPS Spotlight paper with Jan Bauer*, @aproca.bsky.social*, @saxelab.bsky.social, @summerfieldlab.bsky.social, Ali Hummos*! openreview.net/pdf?id=AbTpJ... We study how task abstractions emerge in gated linear networks and how they support cognitive flexibility.
26515
Simon Schug @smonsays.bsky.social · 20/11/2024
Would love to be added as well :)
000
Reposted by Simon Schug
Blake Richards @tyrellturing.bsky.social · 13/11/2024
Great thread from @michaelhendricks.bsky.social! Reminds me of something Larry Abbott once said to me at a summer school: Many physicists come into neuroscience assuming that the failure to find laws of the brain was just because biologists aren't clever enough. In fact, there are no laws. 🧠📈 🧪
4689
Reposted by Simon Schug
Griffiths Computational Cognitive Science Lab @cocoscilab.bsky.social · 18/11/2024
(1/5) Very excited to announce the publication of Bayesian Models of Cognition: Reverse Engineering the Mind. More than a decade in the making, it's a big (600+ pages) beautiful book covering both the basics and recent work: mitpress.mit.edu/978026204941...
14521120
Simon Schug @smonsays.bsky.social · 16/11/2024
🙋‍♂️
010
Simon Schug @smonsays.bsky.social · 13/11/2024
To help find people at the intersection of neuroscience and AI. Of course let me know if I missed someone or you’d like to be added 🧪 🧠 #neuroskyence go.bsky.app/CAfmKQs
335018
Simon Schug @smonsays.bsky.social · 13/11/2024
I think you are already part of it - just double checked :)
010
Simon Schug @smonsays.bsky.social · 28/10/2024
tl;dr: hypernetworks are hiding in our beloved transformers. github.com/smonsays/hyp...
github.com
GitHub - smonsays/hypernetwork-attention: Official code for the paper "Attention as a Hypernetwork"
Official code for the paper "Attention as a Hypernetwork" - smonsays/hypernetwork-attention
010
Simon Schug @smonsays.bsky.social · 28/10/2024
With language being highly compositional itself, could the hypernetwork mechanism play a part in explaining the success of multi-head attention? Maybe! Have a look at the paper in case you are curious! arxiv.org/abs/2406.05816
arxiv.org
Attention as a Hypernetwork
Transformers can under some circumstances generalize to novel problem instances whose constituent parts might have been encountered during training but whose compositions have not. What mechanisms und...
120
Simon Schug @smonsays.bsky.social · 28/10/2024
Indeed in line with the hypothesis that the hypernetwork mechanism supports compositionality, this modification (hyla) improves performance on unseen tasks.
110
Simon Schug @smonsays.bsky.social · 28/10/2024
So what happens if we strengthen the hypernetwork mechanism? Could we maybe further improve compositionality? We can for instance make the value network nonlinear - without introducing additional parameters.
110