Reposted by Bene
⚠️ Stop pretraining your looped Transformer with a separate KV cache per recursion!
In our recent preprint, we show that *memory sharing* not only saves memory but is also a net-positive inductive bias!
Less memory, same flops, higher quality.
🔗 arxiv.org/abs/2610.02383
🧵
