A preprint describes a control mechanism for LLM agents that keeps small routing decisions computed on the GPU instead of sending them back to host memory for redispatch. The authors report faster execution than host-based dispatch across 36 tested placements. The work is positioned as a way to bound GPU opportunity and cut per-step communication overhead in agent control loops.
A new arXiv paper introduces mKernel, a technique that fuses kernels across multiple GPUs and nodes to reduce communication overhead in distributed training and inference. Existing approaches that overlap communication with computation on separate streams only offset part of the cost, so the authors propose kernel-level fusion instead. The work targets the communication bottleneck that limits scaling of large models.
A new arXiv paper examines how device-initiated communication affects fused GPU kernels that issue remote stores straight from their compute pipelines, a common pattern in expert and tensor parallelism. The authors introduce X-Stage, a model of post-issue backpressure that arises once such stores have been dispatched. The work aims to characterize performance limits in these communication-computation overlapped workloads.
A new arXiv preprint examines how to adapt large language models after pretraining when training is spread across consumer-grade GPUs connected by ordinary internet links. The authors focus on the communication overhead that arises along both data-parallel and pipeline-parallel dimensions, which they identify as the main constraint in such decentralized setups. The work targets post-pretraining adaptation rather than training from scratch.
Nvidia has introduced the RTX Pro 5500, a workstation graphics card that shares its design with the GeForce RTX 5090 but offers far more video memory for AI workloads. The company is increasingly positioning gaming as a secondary part of its business. The card is aimed at professionals running large models locally.
Graphics card makers saw unusually strong unit sales in the second quarter, according to a market analysis. The uptick may be partly attributable to demand from AI computing facilities in China. The report does not specify which vendors or regions contributed most to the gain.
A new arXiv paper introduces SparseDitto, a framework that uses an agentic approach to generate sparse matrix computation code for GPUs. It synthesizes implementations that account for both the sparsity pattern of the input and the specifics of the target hardware. The authors note that no single existing implementation performs best across all sparsity patterns, operators, and GPU architectures.
A new arXiv preprint describes JAX-ESHN, a rewrite of the ES-HyperNEAT neuroevolution method using JAX to run whole populations on GPUs. The authors note that, to their knowledge, no existing implementation of the algorithm had offered this level of parallelism before. ES-HyperNEAT itself develops neural network layouts by adaptively subdividing a quadtree, and the new implementation is meant to make it faster and more practical on modern hardware.
A research paper demonstrates that NVIDIA's Blackwell Ultra B300, which trades sharply reduced FP64 vector throughput for much higher FP8 tensor performance, can still execute full double-precision 3-D FFTs by emulating FP64 arithmetic on FP8 tensor cores using the Ozaki Scheme II approach. The authors identify an 'integer-epilogue wall' that limits such emulation and outline minimal hardware changes that would eliminate it.
A new arXiv paper presents GPU-CFR, a technique that compiles game trees into static dataflow graphs so that counterfactual regret minimization can run efficiently on GPUs. The approach uses CUDA graph replay to reduce the overhead of the many small, interdependent memory operations that previously made CPUs faster for this workload. The authors report speedups of up to 80x over existing implementations.
Nvidia has introduced CUDA Rust, allowing developers to write GPU kernels directly in Rust for the first time. Previously, Rust code could only launch CUDA kernels, not define them. The change closes a long-standing gap in Rust's GPU programming support.
A Hugging Face blog post demonstrates how to run reinforcement learning from human feedback on 20-billion-parameter language models using one 24GB consumer graphics card. The write-up covers the techniques that reduce memory requirements enough to make this training approach feasible on hardware most users already own. It is presented as a practical walkthrough rather than a commercial product or model release.
WHY IT MATTERS ↘If RLHF can be run on a single consumer GPU, the cost of experimenting with alignment and post-training methods drops sharply, shifting that work from well-funded labs to individuals and smaller teams. That weakens the assumption that frontier-scale fine-tuning requires datacenter-class hardware, at least for models in the 20B range.