A new arXiv paper introduces CRAF, a cross-view residual-aware fusion approach for detecting synthetic or converted speech. The method targets generalization to spoofing attacks not seen during training, a common weakness in deepfake audio detection. It builds on pretrained speech and audio models.
A new arXiv paper introduces CCMAN, a cross-modal attention model designed to detect early cognitive decline from verbal fluency speech tasks. Unlike prior approaches that pool features over an entire recording, the method explicitly accounts for cognitive instability and aims to produce interpretable temporal biomarkers. The work is framed as a scalable, non-invasive complement to conventional clinical assessment.
A new arXiv preprint introduces ER-EDF, a framework that draws on psychological theories of emotion perception and regulation to guide empathetic responses in spoken dialogue systems built on large audio-language models. The work aims to improve how such systems both recognize a speaker's emotional state and regulate their own generated reply. It is a research contribution and has not been presented as a product or model release.
A new arXiv paper proposes a reference-free way to measure whether text-to-speech systems render lexical tone correctly in languages where pitch changes word meaning. The authors note that standard character error rate scoring misses such errors, using Yorùbá words like "ọkọ" (husband), "ọkọ̀" (vehicle) and "ọkọ́" (hoe), which differ only by tone, as an illustration. The work aims to give a budget-friendly evaluation option for massively multilingual speech systems.
A new arXiv preprint proposes a way to give full-duplex speech-to-speech models the ability to transcribe the user's speech while the conversation is ongoing. Such models can listen and speak at the same time but generally do not produce their own text transcript of what the user says, which limits uses like live captions and conversational records. The work targets streaming transcription so text is generated as the user speaks rather than after the turn ends.
Researchers examined whether visual information from face-to-face interaction improves prediction of conversational turn-taking, since most dialogue systems rely only on audio. The work applies voice activity projection to combine verbal and non-verbal signals. Results suggest visual cues can complement audio-based models of when speakers take turns.
A new arXiv preprint examines how speaker recognition networks organise the latent representations they learn from speech utterances. The work targets the limited transparency of these models by analysing whether speaker embeddings are arranged hierarchically. The authors position the study as a step toward explaining the internal mechanisms behind speaker identification systems.
A new arXiv paper introduces ParaBridge, a method aimed at connecting the paralinguistic information in speech — such as vocal tone, speaker traits, or background noise — with the responses a spoken dialogue system produces. The authors note that while existing speech language models can detect such cues, they often fail to let that perception shape their replies, and the work targets closing that gap.
A new study examines how autoregressive speech recognition systems that output timestamps as decoded tokens can gradually lose alignment during long stretches without speech. The authors propose a replay-based distribution editing approach that corrects this drift while limiting forgetting of previously learned behavior. The work targets timestamped transcription without relying on frame-level aligners or inference-time fixes.
A new arXiv paper introduces CAL-MOS, a method that uses adapters to combine representations from multiple layers of speech foundation models for non-intrusive speech quality assessment. The approach aims to make mean opinion score (MOS) prediction more robust when the underlying foundation model varies. The abstract notes that selecting which layer's representations to use remains an open question in this area.
A new arXiv paper examines why full-duplex speech models such as Moshi and its PersonaPlex derivative sometimes start talking when the user has gone quiet, including on digital-zero input. The authors trace these unwanted onsets to specific causes in the generation process and propose ways to reduce them. The work targets improving turn-taking reliability in speech-to-speech systems.
Researchers released tau-Elicitation, a 200-task benchmark that measures how well voice agents capture specific entities such as names, addresses, identifiers, dates, and times across multi-turn conversations. The set spans ten entity types with controlled difficulty levels, aiming to pinpoint the exact turn where information capture breaks down. The authors argue that end-to-end evaluations hide these failure points, making targeted diagnosis difficult.
Researchers released ParsVoice, described as the largest publicly available multi-speaker Persian speech corpus, aimed at addressing the scarcity of open speech-text data for the language. The dataset is intended to support work on multi-speaker text-to-speech, speech-language modeling, and other low-resource speech tasks. The paper was posted to arXiv under computer science categories.
A revised arXiv paper examines how unsupervised term discovery systems segment unlabelled speech and group the resulting units into candidate word or syllable types. The authors note that real lexicons follow a Zipfian frequency distribution, but the widely used centre-based clustering approach does not reproduce it. Their work introduces a method aimed at recovering that distribution in the discovered lexicon.
A new arXiv paper examines whether reference-free speech quality predictors such as UTMOS, DNSMOS and SCOREQ are dependable both as automatic evaluators for text-to-speech systems and as reward signals in preference optimization. The authors argue that these dual roles rest on assumptions about prediction accuracy that may not hold for modern TTS outputs. The work suggests current evaluation practices could mislead comparisons and reward-based training.
A new arXiv paper addresses a mismatch that arises when speech language models autoregressively generate interleaved text and acoustic tokens. The authors propose a joint-output on-policy distillation approach intended to close this output-mode gap while preserving the streaming and text-guided benefits of the interleaved design. The work falls under computation and language research and has not yet been peer reviewed.
A new arXiv preprint addresses the difficulty of synthesizing nonverbal vocalizations such as laughs, sighs, and coughs in controllable speech generation. The authors attribute the challenge to the acoustic variety of these sounds and their uneven representation in existing speech corpora, and propose modeling, scaling, and decoding methods to improve them. The work is listed under the cs.AI cross-list announcement.
A new arXiv preprint examines connected speech in people with psychosis, framing the condition as a reduced ability to compress information as language unfolds. The authors use large language model embeddings, which represent linguistic input as high-dimensional vectors, to analyze these speech patterns. The work sits within broader efforts to apply LLM-based tools to the study of language in neurodiverse populations.
Researchers introduce CONVTRAP-TN, a transformer-based architecture designed to determine whether two speakers share a first-order family relationship using only audio. The work also releases a new uncontrolled Mandarin kinship speech dataset and reports an ablation study on the model's components. Kinship verification from voice is a relatively underexplored area, and the Mandarin dataset addresses a gap in multilingual resources.
A new arXiv preprint presents an approach that uses conversational audio encoders to spot evasive language from managers during earnings conference calls. Rather than aggregating vocal and lexical features across an entire call, the method analyzes the conversational dynamics between analysts and executives. Prior research has tied such cues to later negative outcomes for firms, and this work aims to capture them more precisely.
A new arXiv paper proposes a way to quantify the quality gap between speech-only language models and those that also handle text. The authors note that the gap is hard to measure because speech and text systems are usually trained on different data and judged with different metrics. The method aims to put the two modalities on a comparable footing.
A new arXiv paper examines how well audio-language models can turn long doctor-patient conversations into structured SOAP clinical notes. The authors compare lightweight and heavyweight end-to-end approaches, noting that while cascaded speech recognition pipelines remain strong, end-to-end models tend to lose information or produce hallucinations. The work targets the modality gap in long-form clinical audio.
A research paper examines how speech and audio LLMs are typically evaluated, namely by testing whether a waveform prompt outperforms an automatic speech recognition transcript. The authors argue that for closed-set, known tasks this setup mixes together two distinct things: whether the model actually used acoustic information and whether it simply needed the task spelled out. They propose auditing generative audio calls as a way to separate those factors.
New arXiv research investigates AI models that detect early signs of dementia and mild cognitive impairment from conversational speech, focusing on multilingual English speakers in the UK. The authors report that such screening tools show a false positive bias, disproportionately flagging multilingual speakers compared with monolingual ones. The findings suggest speech-based cognitive screening may disadvantage linguistically diverse populations unless corrected.
Researchers have released S3-Bench, a benchmark aimed at measuring how well speech interaction models function as voice assistants for scientific work. The evaluation focuses on multimodal large language models, examining whether their conversational strengths extend beyond general-purpose assistant tasks to domain-specific spoken interactions.
A new arXiv paper introduces NOPE-HYPE, a structured simulation workflow for examining how speech-to-text translation systems perform under a wide range of acoustic conditions. The authors argue that large speech models remain sensitive to environments they have not encountered and that current pipelines lack controllable tools for exploring such scenarios. The workflow offers researchers a systematic way to probe model robustness before deployment.
Researchers introduce StreamAlign, a speech tokenization method that maps audio into tokens aligned with large language model token spaces while operating in a streaming, low-latency manner. Unlike existing text-aligned tokenizers that depend on offline automatic speech recognition, the approach addresses the latency and alignment limitations that offline processing imposes, enabling more efficient use of pretrained LLMs for speech tasks.
A new arXiv paper introduces a unified pipeline that uses a single phoneme-based text-to-speech model to generate synthetic training data for automatic speech recognition. The work presents a controlled study of how text selection, reference speech, and augmentation scale affect recognition accuracy.
Researchers introduce X2-NativeCursor, a mechanism that lets incremental-text streaming TTS systems track which part of the input is currently being spoken. Since text usually arrives ahead of the audio, arrival timing alone cannot indicate spoken progress, so the method provides a token-level cursor for alignment. This supports features like synchronized highlighting, interruption handling, and real-time dialogue-history updates in speech applications.
Researchers have released a study disentangling two distinct gender signals that speech-to-speech models can pick up: the acoustic characteristics of a speaker's voice and gender-related cues embedded in what is being said. This distinction matters for applications like dubbing, translation, and voice agents, where an ideal system should preserve how a speaker actually sounds rather than defaulting to stereotyped content. The work offers a framework for auditing whether these models rely on voice or on content-based assumptions when producing gendered output.
Researchers introduce an approach for determining how speakers in spoken dialogues relate to one another, framing the task as a step toward socially aware speech understanding. Rather than costly supervised training that is difficult to scale, the method relies on multiple cooperating reasoning agents. The paper targets a task the authors describe as still largely underexplored in speech and language research.
Researchers have introduced SEA-SpeechBench, a large-scale multitask benchmark for evaluating how well speech and multimodal language models handle Southeast Asian languages. The paper argues that existing evaluation frameworks for audio-capable models focus heavily on English, leaving the region's languages poorly covered. The benchmark aims to enable broader assessment of speech understanding performance across the area's many languages.
A new arXiv study investigates self-supervised speech foundation models, which are widely deployed for speech recognition and to supply tokens for speech-capable language models. The authors analyze what these models' internal representations encode, testing whether they capture genuine word-level linguistic structure rather than only acoustic patterns. The results bear on how such models should be interpreted and used in downstream speech applications.
Researchers present V2TATC, a machine-learning approach that jointly embeds controller voice communications and aircraft trajectory data to support situational awareness in air traffic control. The work also introduces a new dataset to enable development of scalable decision-support tools as traffic, particularly at low altitudes, grows in the US National Airspace System.
A new research paper notes that most voice-clone detectors output only a single confidence score per audio clip, leaving their verdicts difficult to inspect or trust. The authors propose moving from raw scores to decisions supported by auditable evidence, and report that this evidence-oriented design can also boost detection performance. The work targets speech deepfakes that convincingly imitate real speakers' voices.
A new arXiv preprint presents a fine-tuned ByT5 model that converts Filipino text into phoneme sequences at the sentence level while also predicting stress. Since Filipino's spelling maps to sounds fairly predictably, the authors focus on the harder problem of capturing prosody, which they tackle using weakly supervised training data.
A new arXiv preprint introduces ZipCodec, a neural audio codec aimed at streaming speech at very low frame rates. The authors note that pushing bitrates down is easier than lowering frame rate, since fewer tokens per second means each token must carry more information. The work targets the trade-off between reconstruction quality and the amount of information each token encodes.
A new arXiv preprint argues that word error rate alone hides important failures when speech recognition systems and audio language models handle code-switched speech. The authors propose an evaluation method that accounts for language switches, and apply it to English-Yoruba audio, a low-resource pair with diacritics. They find that strong monolingual benchmark scores do not carry over to this setting.
A new arXiv paper investigates how much different aspects of speech contribute to detecting when a speaker has finished their turn in a conversation. The authors run a controlled ablation of a conversational system to separate the relative weight of each modality, noting that the role of semantics versus other cues is still poorly understood. The work targets more natural turn-taking in conversational AI.
A new arXiv preprint looks at how neural networks in speaker recognition pick up patterns beyond those explicitly defined by researchers, treating these as "second-order" patterns. The work connects classical pattern recognition training with explainable AI methods that surface latent features underlying a network's decisions. It focuses on the speaker recognition domain as a case study.