On-Policy Self-Distillation Method Aims to Prevent Entropy Collapse in RL-Trained LLMs

A new arXiv paper proposes an approach called on-policy self-distillation that acts as a "policy reheater" for reinforcement learning with verifiable rewards. The authors target entropy collapse, a failure mode where model policies become overly concentrated, cutting rollout diversity and weakening the learning signal. The method is presented as a way to keep exploration alive during RL training of large language models.