"Learning to Reason in 13 parameters"
An interesting paper by Meta: arxiv.org/abs/2602.04118
Authors propose "TinyLoRA". They showed that Qwen 8B parameter model was able to achieve about 92% accuracy on certain maths datasets with just "13 trainable parameters (26 bytes)" for finetuning.
#ML
arxiv.org
Learning to Reason in 13 Parameters
Recent research has shown that language models can learn to \textit{reason}, often via reinforcement learning. Some work even trains low-rank parameterizations for reasoning, but conventional LoRA can...