iiinova — where your digital self keeps evolving

Gabriel Béna 🌻

@solarpunkgabs.bsky.social

407 followers 401 following 95 posts

Postoc at Insitute of Neuro-Informatics in Zurich, with Melika Payvand. Pretending to be a neuro guy. Working on self-organising and self-healing hardware. Trying to make biology re-emerge through genomic rules in silicon.

Posts Replies Media

Gabriel Béna 🌻 @solarpunkgabs.bsky.social · 19/08/2026

We mapped 50,000 healed circuits — all computing the same function, all perfect. SODC (left): a galaxy of distinct solution "archetypes”, a whole neutral landscape. Backprop (right): one lonely blob. Degeneracy, visualized.

Two UMAP scatter plots embedding 50,000 circuit configurations that all compute the same function perfectly (UMAP dimension 1 on x, dimension 2 on y). Points are coloured by how many perturbation–recovery cycles they are from the root circuit, which is marked with a red star; thin grey lines link successive states. Left ("TMT", the local policy): the points organise into many separate clusters spread across the whole space — a landscape of distinct structural "archetypes". Right ("BP", backpropagation): a single dense blob around the root. Same task, same accuracy — but the local policy explores a far more degenerate solution space.

110

Gabriel Béna 🌻 @solarpunkgabs.bsky.social · 19/08/2026

And also, it's scale-free in the best way: train on tiny 264-gate circuits, deploy on 450+ gates it's never seen… and accuracy goes UP 📈 Turns out "learn a local rule" generalizes way better than "memorize a big circuit." Who knew. (biology knew.)

Two line charts of hard accuracy (y-axis, 0.5–1.0) versus circuit width in number of nodes (x-axis, ~170–450). A dashed vertical line marks the training size of 264 nodes; green is evaluated without damage, blue with damage. Left ("fixed wiring"): accuracy spikes to ~1.0 exactly at the 264-node training size and collapses to about 0.5 on either side — the policy overfits the scale it trained on. Right ("random wiring"): accuracy stays high (~0.8–0.9) across all widths and even rises as the circuit grows past the training size, with shaded confidence bands — scale-freedom is a learned capability.

110

Gabriel Béna 🌻 @solarpunkgabs.bsky.social · 19/08/2026

So naturally, we shot the circuit. Repeatedly. 🔫 The wild part: when a gate dies, SODC does not restore the original. It hops to a different configuration that computes the same function. Repair ≠ rewind. Repair = find another way.

A grid of PCA trajectory plots — principal component 1 on the x-axis, component 2 on the y-axis — tracing how a circuit's configuration moves while the local policy repairs it. Line colour runs from dark blue at step 0 to yellow at step ~250. Columns: no damage, stochastic failures, shotgun failures; rows: recoverable vs permanent damage. With no damage the trajectory is a single clean arc. Under damage the paths fan out and settle in different places — the policy re-routes to new, functionally-equivalent configurations instead of rewinding to the original.

110

Gabriel Béna 🌻 @solarpunkgabs.bsky.social · 03/07/2025

We'll be presenting this at #GECCO2025!! Come say hi if you're around ☀️

061

Gabriel Béna 🌻 @solarpunkgabs.bsky.social · 05/06/2025

The REAL question on everyone's lips though... Blog: gabrielbena.github.io/blog/2025/be... Thread: bsky.app/profile/sola...

0123

Gabriel Béna 🌻 @solarpunkgabs.bsky.social · 04/06/2025

Taking it even further: We're developing a graph-based "Hardware Meta-Network"! Users define tasks as intuitive graphs (nodes = regions, edges = operations), and a GNN + coordinate-MLP generates the hardware configuration! It's literally a compiler from human intent → NCA computation! 🤖

Graph-based task representations and GNN-based hypernetwork for hardware generation

110

Gabriel Béna 🌻 @solarpunkgabs.bsky.social · 04/06/2025

Our approach also enables task composition, meaning we can chain operations together! Example: Distribute matrix → Multiply → Rotate → Return to original position It's like programming, but the "execution" is continuous dynamics! We're building a neural compiler!

Examples of an Out of Distribution Task: the NCA needs to distribute a matrix in all corners in a larger grid than the one seen during training.

100

Gabriel Béna 🌻 @solarpunkgabs.bsky.social · 04/06/2025

More on the MNIST demo: We pre-train a linear classifier, decompose the 784×10 matrix multiplication into smaller blocks, and let the NCA process them in PARALLEL! Emulated accuracy: 60% (vs 84%), not perfect due to error accumulation, but it WORKS! This is a neural network running inside a CA! 🤯

NCAs as a neural network emulator. Inputs show a batch of flattened MNIST images, alongside the weight matrix of a pre-trained single-layer linear classifier. We decompose this matrix multiplication into sub-blocks, that can be directly emulated in parallel by the NCA. Results are fetched from NCA states and aggregated back into logits, of size batch x 10 (figure shows the first 32 outputs only for readability). We compute accuracy by taking the logits argmax per batch and comparing with labels.

150

Gabriel Béna 🌻 @solarpunkgabs.bsky.social · 04/06/2025

Through this framework, we are able to successfully train on a variety of computational primitives of matrix arithmetics. Here is an example of the NCA performing Matrix Translation + Rotation directly in its computational state (and, by design, only using local interactions to do so) !

130

Gabriel Béna 🌻 @solarpunkgabs.bsky.social · 04/06/2025

We propose a novel framework that disentangles the concepts of “hardware” and “state” within the NCA. For us: - Rules = "Physics" dictating state transitions. - Hardware = Immutable + heterogeneous scaffold guiding the CA behaviour. - State = Dynamic physical & computational substrate.

Our framework performs matrix operations (translation, multiplication, rotation) through continuous cellular interactions evolving over time (from t=0 to t=60). Core mechanism: cells communicating only with neighbors, and interacting with a local fixed (but learned) heterogeneous substrate, are able to collectively solve mathematical tasks without explicit algorithms.

161

Gabriel Béna 🌻 @solarpunkgabs.bsky.social · 28/01/2025

Another exciting part? We developed a parallel *fully identical* off-chip simulator, opening doors for hybrid training approaches. This is particularly exciting for online learning scenarios, where networks train on continuous data streams - crucial for embedded systems and autonomous agents. (6/8)

Accuracy comparison between the on and off chip implementations, for both the full dataset or the reduced dataset. Both show minimal differences and good accuracy.

120

Gabriel Béna 🌻 @solarpunkgabs.bsky.social · 28/01/2025

Our Solution: We implemented EventProp on SpiNNaker2, computing exact gradients through sparse error signal communication between neurons. The key innovation? Maintaining temporal sparsity during both forward AND backward passes - a significant departure from traditional BPTT. (4/8)

Software and Hardware stack of the implementation.

110