A German-language paywalled article examines what determines the price of running different AI tasks on language models. It breaks down the cost factors behind model usage and points to places where users can cut spending most effectively.
A new arXiv preprint examines whether a language model can act as a compact, living specification that answers detailed questions while avoiding the tradeoff between vague specs and lengthy ones. The work frames specifications as needing to balance leaving details out against recording every detail separately. The paper appears in the cs.LG category as a new submission.
A new arXiv paper examines the role of language in multimodal systems, noting that language models use text as input, output, and an increasingly internal representation. The author argues that whether language should hold all of these positions depends on what language does to the system that relies on it, drawing on evidence from human perception and cognition.
A new arXiv paper argues that research on negation in language models has focused too narrowly on a few common single-word cues such as "not" and "never". The authors examine a wider range of negation signals and report that affixal negations, formed through prefixes and suffixes, lead to better negation understanding than the cues usually studied. The work points to broader cue coverage as a way to address a persistent weakness in both LMs and LLMs.
A newly revised arXiv paper proposes a perturbation-based technique for locating and evaluating linguistic representations inside deep neural language models, framing it as an adversarial tracer. The authors note that representation discovery remains unresolved, and that loosely constrained alignment procedures can make the very notion of a representation vacuous. Their approach aims to provide a simpler and more efficient way to probe how such models encode language.
New arXiv research explores how language models resist harmful behavior shifts caused by malicious fine-tuning. The work evaluates preventative steering, a training-time method that injects undesirable persona vectors during fine-tuning and removes them at inference, and analyzes how the defense's effectiveness changes over the course of training. The findings suggest these safeguards require active adjustment across training phases rather than a fixed configuration.
An arXiv research paper questions why dense Transformers almost universally place most of their non-embedding parameters in narrow-wide-narrow feed-forward networks. Drawing on theoretical and empirical evidence, the authors explore an alternative wide-narrow-wide (hourglass) residual design for these blocks. The work is cross-listed across the cs.AI, cs.CL, and cs.LG categories on arXiv.