A new arXiv preprint describes a method for letting stakeholders define reward functions for reinforcement learning agents without needing to sample from the environment. The authors position the work as reducing the manual effort of reward design that preference-based approaches like online RLHF are meant to address. The same paper was listed in both the cs.AI and cs.LG announcement feeds.
A new arXiv paper in machine learning introduces ESSA, which uses evolutionary strategies as an alternative to gradient-based RLHF methods like PPO and GRPO for aligning large language models. The authors argue that existing pipelines are costly because they require backpropagation through long rollouts, and their approach avoids this bottleneck. The work targets more scalable online alignment of LLMs.
A Hugging Face blog post demonstrates how to run reinforcement learning from human feedback on 20-billion-parameter language models using one 24GB consumer graphics card. The write-up covers the techniques that reduce memory requirements enough to make this training approach feasible on hardware most users already own. It is presented as a practical walkthrough rather than a commercial product or model release.
WHY IT MATTERS ↘If RLHF can be run on a single consumer GPU, the cost of experimenting with alignment and post-training methods drops sharply, shifting that work from well-funded labs to individuals and smaller teams. That weakens the assumption that frontier-scale fine-tuning requires datacenter-class hardware, at least for models in the 20B range.