iiinova — where your digital self keeps evolving

Trinity Chung

@trinityjchung.com

22 followers 21 following 9 posts

MS @ Carnegie Mellon University Robotics Institute BS @ UC Berkeley Computer Science

Posts Replies Media

Trinity Chung @trinityjchung.com · 27/05/2025

6/ We found that ConvRNNs (esp. IntersectionRNNs of @jascha.sohldickstein.com @sussillodavid.bsky.social ) beat ResNets, Transformers, and SSMs when it comes to aligning with real mouse somatosensory cortex data. Plus, they pass the NeuroAI Turing Test! (bsky.app/profile/anay...)

Comparison of neural fit (noise-corrected RSA Pearson’s r) across models. The mean animal-to-animal score is 0.18 and the maximum between all pairs of animals is 1.16. The leftmost “a2a” bar represents the mean animal-to-animal neural consistency score. The lighter-colored left bar represents the randomly initialized version for every model.

120

Trinity Chung @trinityjchung.com · 27/05/2025

5/ Crucially, we found that when doing tactile self-supervised learning, we couldn’t apply all of the usual image augmentations out-of-the-box (in fact, they couldn’t train at all!), so we designed transformations explicitly for tactile force & torque inputs:

Types of data augmentations applied to SSL models. Given a temporal tactile input over time T, our tactile augmentation vertically, horizontally, temporally flips, and rotates the features, while traditional image augmentation introduces Gaussian noise, color jitter, and grayscale. The tactile augmentations were effective in improving both the neural fit and task performance. The models were unable to be trained with image augmentations.

120

Trinity Chung @trinityjchung.com · 27/05/2025

4/ We trained our models using data from simulated mouse whiskers from @MitraHartmann's group brushing various objects. Whiskers realistically output detailed force-torque signals, unlike current robot sensors, which remain difficult to scale and have reduced sensitivity in sim.

We use average mouse whisker array measurements to recreate a whisker model in sim. (II) Objects are whisked in simulation using WHISKiT (from Hartmann's lab) resulting in (III) force and torque data for sweeping 9981 ShapeNet objects of 117 categories with various sweep augmentations. The augmentations vary the (1) speed, (2) height, (3), rotation, and (4) distance of the objects relative to the whisker array. We constructed two datasets: a large, low-fidelity set with more sweep augmentations, and a small, high-fidelity set with fewer augmentations.

120

Trinity Chung @trinityjchung.com · 27/05/2025

3/ We systematically explored temporal architectures (unifying ConvRNNs including from @Chengxu & @dyamins.bsky.social et al. 2017's prior tactile work, SSMs, Transformers) via our Encoder-Attender-Decoder (EAD) framework, built using a custom "PyTorchTNN" library we developed.

The Encoder-Attender-Decoder (EAD) architecture, with task objectives being supervised categorization, self-supervised learning (SimCLR, SimSiam, autoencoding). The ConvRNN encoder includes self-recurrence at each layer where we vary different RNNs.

120

Trinity Chung @trinityjchung.com · 27/05/2025

1/ What if we make robots that process touch the way our brains do? We found that Convolutional Recurrent Neural Networks (ConvRNNs) pass the NeuroAI Turing Test in currently available mouse somatosensory cortex data. New paper by @Yuchen @Nathan @anayebi.bsky.social and me!

Task-Optimized Convolutional Recurrent Networks Align with Tactile Processing in the Rodent Brain

1185