arxiv.org
N-Gram Induction Heads for In-Context RL: Improving Stability and Reducing Data Needs
In-context learning allows models like transformers to adapt to new tasks from a few examples without updating their weights, a desirable trait for reinforcement learning (RL). However, existing in-co...