Zone of Proximal Policy Optimization: teacher guidance via prompts, not gradients

A new arXiv paper argues that knowledge distillation breaks down when the student model is much smaller than its teacher, because imitating the teacher's logits locks the student into its sharpest output modes and harms generalization. The authors propose letting the large teacher guide the small student through prompts during reinforcement-learning fine-tuning instead of through gradient-based distillation. The work appears in the computational linguistics category on arXiv.