A revised arXiv paper titled Mimir argues that current language modeling is organized around tokens, where corpora are split into tokens and models are trained on token-level objectives such as next-token prediction. The authors propose an alternative that works with concepts at a large multilingual scale. The submission is a replacement version of a cross-listed paper.
A new arXiv paper introduces DA-DLM, a method for diffusion language models that explicitly captures relationships between tokens during generation. Existing diffusion models denoise masked text by predicting several tokens at once under an assumption of conditional independence, which the authors say weakens coherence. The proposed approach aims to restore those inter-token dependencies.
Researchers introduce URCHIN, a spiking neural language model designed for pretraining on small, developmentally plausible text corpora, as targeted by the BabyLM challenge. The work argues that most existing language models ignore the biological properties of the neural circuitry that underlies human language acquisition. It tests how much language a model can learn from child-scale data instead of internet-scale datasets.
A new arXiv paper addresses a mismatch that arises when speech language models autoregressively generate interleaved text and acoustic tokens. The authors propose a joint-output on-policy distillation approach intended to close this output-mode gap while preserving the streaming and text-guided benefits of the interleaved design. The work falls under computation and language research and has not yet been peer reviewed.
A revised arXiv preprint analyzes how language models handle three forms of reasoning: deduction, induction, and abduction. The authors compare ways tasks are specified to models, such as explicit instructions versus few-shot examples, and argue that current evaluations leave parts of the reasoning picture unresolved. The work is a research paper rather than a product or model release.
A revised arXiv paper examines why masked diffusion language models can produce incoherent text blocks: they decode every position in a block in parallel from separate marginal distributions. The authors propose a coupled-noise distillation approach intended to let such models generate a whole block in a single step while keeping the tokens mutually consistent. The work is a research preprint and has not been peer-reviewed.
A new arXiv paper examines the role of language in multimodal systems, noting that language models use text as input, output, and an increasingly internal representation. The author argues that whether language should hold all of these positions depends on what language does to the system that relies on it, drawing on evidence from human perception and cognition.
A new arXiv paper proposes a way to quantify the quality gap between speech-only language models and those that also handle text. The authors note that the gap is hard to measure because speech and text systems are usually trained on different data and judged with different metrics. The method aims to put the two modalities on a comparable footing.
A new arXiv preprint addresses the limited ability of large audio-language models to pinpoint when specific sounds occur within a recording. The authors propose adding frame-level grounding during training so these models can localize audio events more precisely rather than only describing clips in broad terms. The work targets fine-grained temporal understanding, a known weak point for current audio-language systems.
A new arXiv preprint introduces discrete Beckmann transport models, a framework for language modeling and reasoning that avoids the multi-step sampling usually required by discrete diffusion and flow approaches. The authors note that existing methods compress sampling steps only by distilling a pretrained autoregressive teacher, which limits the student to the teacher's performance. Their approach instead aims to generate text in a single step without relying on that distillation ceiling.
A new arXiv paper revisits comparisons between state-of-the-art coreference resolution systems. Because every leading system fine-tunes a pretrained language model, the authors ask whether differences in scores come from the underlying language model or from task-specific design choices. The work presents a controlled reevaluation to separate those factors.
A new preprint develops a Bayesian account of how intelligence can be inferred from the observable behavior of agents such as language models. The authors treat each prompt as a possibly imperfect internal experiment, and the abstract is truncated before the full results are described. The work sits in the broader area of evaluating model capability without direct access to internal states.
A new arXiv paper argues that research on negation in language models has focused too narrowly on a few common single-word cues such as "not" and "never". The authors examine a wider range of negation signals and report that affixal negations, formed through prefixes and suffixes, lead to better negation understanding than the cues usually studied. The work points to broader cue coverage as a way to address a persistent weakness in both LMs and LLMs.
Researchers released MameLoshnLM, described as the first open-source 8-billion-parameter language model dedicated to Yiddish. The work also includes an evaluation benchmark intended to fill the gap in reliable testing resources for the language. It addresses the low digital availability of Yiddish text despite its substantial written heritage.
A new arXiv paper examines how language model priors can be leveraged to craft acoustic adversarial attacks against automatic speech recognition systems. It focuses on real-time ASR, where transcription decisions must be made under strict temporal limits using incomplete audio input. The work suggests that this causal constraint creates an exploitable vulnerability in streaming recognition pipelines.
A new arXiv preprint argues that current neuroscience and language-model research mostly checks whether brain signals or model layers can predict annotated linguistic variables, which shows correlation but not how language is actually implemented. The author proposes instead deriving what a neural system must be capable of from the formal properties of language itself, then treating those requirements as constraints on neural dynamics. This reframes the goal from prediction accuracy toward identifying the mechanisms a system needs in order to support language.
Researchers examined looped variants of GPT-BERT for the BabyLM 2026 shared task, in which a compact set of layers is executed repeatedly rather than stacking many distinct ones. Their findings suggest that reapplying a small parameter budget can match the performance of larger models when training data is scarce. The work positions recurrence as a compute-for-parameters trade-off for building efficient low-resource language models.
A new arXiv paper examines word order preferences in decoder-only language models, testing 192 artificial languages alongside typologically diverse natural languages. The authors report a consistent left-branching bias in the models and argue that training data, rather than linguistic typology, shapes these preferences. The study also explores how this bias relates to model performance on right-branching languages.
A new arXiv paper introduces CARTS, a method that uses autoregressive language models to hide a payload text inside a stegotext of the same token length. It preserves per-position rank information across contexts, which the authors formalize as Contextual Autoregressive Rank Transcoding. The approach targets full-capacity keyed text encoding, meaning the hidden message can occupy the entire token sequence rather than a limited subset.
A newly posted arXiv paper lays out the full technical approach behind the KuaiRP family of role-playing models. The authors say the design targets four goals, including lighter prompt engineering, more consistent output quality, and built-in domain capabilities. Further implementation and evaluation details were not included in the available excerpt of the report.
A new arXiv paper introduces ChronoBerg, a resource designed to capture how language changes over time and to help foundation models reason about temporal context. The authors argue that while existing training corpora are broad, they often lack the long-term chronological structure needed for time-aware language understanding. The work is posted as a cross-list replacement on arXiv cs.AI.
A newly revised arXiv paper proposes a perturbation-based technique for locating and evaluating linguistic representations inside deep neural language models, framing it as an adversarial tracer. The authors note that representation discovery remains unresolved, and that loosely constrained alignment procedures can make the very notion of a representation vacuous. Their approach aims to provide a simpler and more efficient way to probe how such models encode language.
A new arXiv paper revisits Plaid, a likelihood-based continuous diffusion model for text, to test the assumption that continuous diffusion scales worse than discrete alternatives. The authors report that with the right design, continuous diffusion can match discrete diffusion at scale. The work is a replacement version of a cross-listed machine learning preprint.
Researchers have introduced Edu-QuRating, a data-filtering method for language-model pre-training that evaluates educational value across multiple dimensions instead of a single scalar score. The approach relies on distilled pairwise judgements to rank documents, giving finer-grained control when curating training corpora. The work argues that one-dimensional educational quality metrics can be too coarse for datasets with specific application needs.