doi.org
Random network structure stabilizes neural manifolds
Neuronal activity patterns change continuously over days and weeks, a phenomenon known as representational drift. Despite this, the geometric structure of population representations, namely the pairwise similarities between stimulus-evoked activity patterns, remains remarkably stable. How can ongoing changes in activity be consistent with stable representational similarity? We show that this is a generic consequence of random connectivity: in networks with random connectivity, output similarity is a monotonically increasing function of input similarity, independent of the specific connectivity pattern. Drift, whether driven by random synaptic turnover or Hebbian plasticity, merely transitions the network between random instantiations, leaving similarity intact. This extends to recurrent architectures and to deep neural networks, where continued training beyond performance saturation produces activity drift while preserving representational similarity. Although connectivity in the brain is not random, networks trained on high-dimensional inputs acquire connectivity that behaves statistically like a random projection, making these results broadly applicable to biological neural circuits. ### Competing Interest Statement The authors have declared no competing interest. Spanish Ministry of Science and Innovation, PCI2023-145967-2, PID2021-124702OB-I00 Spanish State Research Agency (AEI) – Severo Ochoa and María de Maeztu Program for Centers and Units of Excellence in R&D, CEX2020-001084-M German Research Foundation (DFG), FOR 5368 ARENA
![Overview of the paper.
Title: The Emergent Symbolic Structure of Artificial Neural Networks
Authors: Tom McCoy, Paul Soulos, Tal Linzen, Paul Smolensky
Left: Neural networks encode information in vectors (there is then an image of a vector), yet they excel at tasks long thought to require symbolic structure (there is then an image of a symbolic representation, specifically a syntax tree). How do LLMs do it?
Right: We find that LLM representations can be closely approximated with symbolic structures. This approximation lets us edit the structure of an LLM’s output by editing the structure of its internal representations, as shown. There is then an image of two edits to LLMs. In the first one, the original input is 3 + 6 * 8, with an answer of 51. But if we swap the positions of the 3 and the 6, the output becomes 30. In the second one, the original input is a Python command repeating the list [Z, U] three times, producing [Z, U, Z, U, Z, U]. But if we edit the input in a way that adds a Q at the end of the input, the output becomes [Z, U, Q, Z, U, Q, Z, U, Q].](https://cdn.bsky.app/img/feed_thumbnail/plain/did:plc:bcsvlv7ubeagj5virdvatadd/bafkreic3zm2rpc4vtqrz2t3y4vcl3t2owp2sxay42vuh5zm5zkg4jhr64e)
