A new arXiv paper examines how post-training choices and test-time inference setups influence a model's ability to write natural-language proofs for difficult olympiad problems. Using Nemotron 3 Ultra as a base, the authors produce two specialist checkpoints via supervised fine-tuning and reinforcement learning, and release the training approach publicly.
A new arXiv paper tackles a weakness in adapting Mixture-of-Experts models: routing layers can destabilise during supervised fine-tuning, causing rarely used experts to lose their specialised knowledge. The authors introduce a tuning approach designed to retain this long-tailed expert information and compare it with earlier anti-collapse techniques such as DenseMixer and ESFT. The work addresses a practical bottleneck for teams adapting large MoE models to downstream tasks.
A new arXiv paper argues that standard supervised fine-tuning applies its loss uniformly across all tokens, even though some are already mastered and others carry far more useful learning signal for mathematical reasoning. The authors introduce a token-trimming perspective that prioritizes which tokens a model should actually learn during fine-tuning, aiming to avoid over-sharpening well-understood tokens while strengthening the ones that matter most.
An arXiv paper introduces a post-training method that blends supervised fine-tuning with reinforcement learning by averaging the output logits of a frozen reference policy with those of the trainable policy. The technique is integrated into GRPO, letting a model retain its supervised behaviors while still improving on verifiable rewards during reinforcement learning.