A revised arXiv paper examines a failure mode in which large language models lose reasoning ability on out-of-distribution tasks as model depth grows. The authors report that giving models access to tools mitigates this depth-related collapse, and they note that OOD generalization remains hard to measure reliably.
A new arXiv preprint argues that strong benchmark performance does not by itself show that a model has learned compositional, environment-invariant features. The authors aim to disentangle measured accuracy from the underlying representations that support out-of-distribution generalisation, a capability often treated as a marker of biological intelligence. Their analysis is framed around how systems can transfer invariant properties from training mappings to novel compositions.
A new arXiv paper introduces Robusto-2, a benchmark evaluating both humans and vision-language models on autonomous driving tasks in Lima, Peru and New York City. The work targets how well multi-modal systems generalize when deployed in unfamiliar, out-of-distribution urban environments. It is a cross-listed replacement submission on arXiv cs.AI.
A revised arXiv paper introduces a benchmark designed to test AI agents on tasks outside the well-known applications that dominate current evaluations. The authors contend that testing in familiar, comparatively simple settings can mask how poorly agents generalize to novel situations. The work aims to give a more accurate picture of how agentic systems will behave in real-world deployment.