Salesforce has introduced Koa, an enterprise-focused language model created by post-training the open-weight Nemotron-3-Super-120B foundation model. The training process uses reinforcement learning with Group Relative Policy Optimization (GRPO), drawing on public data and other sources. The work targets agentic tool use in enterprise settings, according to the arXiv preprint.
A new arXiv paper examines how group-relative policy optimization methods such as GRPO drop rollout groups that show no contrast, through a dynamic sampling step, while real implementations let users configure the filter metric. The authors identify and measure a mismatch between that metric and the predicate used to decide which groups to discard, describing resulting "phantom advantages" when rewards are shaped. The work argues the filter metric choice is safety-critical rather than a minor implementation detail.
An arXiv paper introduces a post-training method that blends supervised fine-tuning with reinforcement learning by averaging the output logits of a frozen reference policy with those of the trainable policy. The technique is integrated into GRPO, letting a model retain its supervised behaviors while still improving on verifiable rewards during reinforcement learning.
A new Hugging Face tutorial demonstrates using GRPO, a reinforcement learning technique, to fine-tune a small 350M-parameter model so it reliably generates valid structured outputs such as JSON. The walkthrough shows that roughly 100 training steps are enough to meaningfully improve format adherence, and it includes code for reproducing the results with open-source tooling.
WHY IT MATTERS ↘Format adherence for structured outputs like JSON is a persistent production bottleneck, and showing that ~100 GRPO steps fix it on a 350M model means teams can handle such workloads with tiny, cheaply trainable local models instead of frontier APIs. That lowers inference costs and latency, enables on-device deployment, and reduces dependence on vendor-gated structured-output features.