We propose an alternative to symbol-permutation data augmentations for structured problem-solving tasks: By introducing a dedicated symbol axis into the recurrent state tensors of RRMs and using an attention-based design, SE‑RRMs can inherently preserve symbol eq. when needed.
