A new arXiv preprint examines how large audio-language models can be made to forget sensitive information they may have memorized during training. The work focuses on the speech question-answering setting, where such models have shown strong performance but also carry privacy risks. The authors frame machine unlearning as a way to reduce unintended retention of private data in these systems.
A new study examines how audio language models attribute speech to the correct speaker, finding accuracy of only 6 to 16 percent on a six-speaker task, below random guessing. The authors show that speaker tracking relies on attention heads in the model's text backbone, and that altering a subset of those heads shifts which speaker the model retrieves.
A new arXiv paper examines how well audio-language models can turn long doctor-patient conversations into structured SOAP clinical notes. The authors compare lightweight and heavyweight end-to-end approaches, noting that while cascaded speech recognition pipelines remain strong, end-to-end models tend to lose information or produce hallucinations. The work targets the modality gap in long-form clinical audio.
A new arXiv preprint addresses the limited ability of large audio-language models to pinpoint when specific sounds occur within a recording. The authors propose adding frame-level grounding during training so these models can localize audio events more precisely rather than only describing clips in broad terms. The work targets fine-grained temporal understanding, a known weak point for current audio-language systems.
A new arXiv preprint introduces ER-EDF, a framework that draws on psychological theories of emotion perception and regulation to guide empathetic responses in spoken dialogue systems built on large audio-language models. The work aims to improve how such systems both recognize a speaker's emotional state and regulate their own generated reply. It is a research contribution and has not been presented as a product or model release.
A new arXiv paper addresses hallucination in large audio-language models, where models sometimes let language priors override what is actually heard in the audio. The authors propose a method that adaptively selects perturbations for contrastive decoding, a training-free approach, arguing that existing techniques rely on crude perturbations such as masking or added noise. The work aims to improve how reliably these models ground their outputs in acoustic evidence.
A new arXiv preprint argues that word error rate alone hides important failures when speech recognition systems and audio language models handle code-switched speech. The authors propose an evaluation method that accounts for language switches, and apply it to English-Yoruba audio, a low-resource pair with diacritics. They find that strong monolingual benchmark scores do not carry over to this setting.