I still uncertain about the RL aspect in DeepSeek.
To me it looks like a clever way of applaying a PPO-like clipping within a supervised framework, constrained by a fixed reference model. Althought some parts in its formulation are very similar to PPO, I wouldn't describe it as RL. (1/5)🧵
