A new arXiv paper examines why full-duplex speech models such as Moshi and its PersonaPlex derivative sometimes start talking when the user has gone quiet, including on digital-zero input. The authors trace these unwanted onsets to specific causes in the generation process and propose ways to reduce them. The work targets improving turn-taking reliability in speech-to-speech systems.
A new arXiv paper studies streaming omni-modal systems that process video chunks alongside synchronized audio and must choose what to respond to and at which moment. The authors note that visual cues can support an interpretation before a spoken utterance or sound event has finished, which complicates response timing. The abstract is truncated, but references a memo mechanism tied to when an interpretation is formed.
A new arXiv paper investigates how much different aspects of speech contribute to detecting when a speaker has finished their turn in a conversation. The authors run a controlled ablation of a conversational system to separate the relative weight of each modality, noting that the role of semantics versus other cues is still poorly understood. The work targets more natural turn-taking in conversational AI.
A new arXiv paper proposes RelayS2S, a dual-path speculative generation method for real-time spoken dialogue systems. It addresses the trade-off between latency and response quality, since end-to-end speech-to-speech models can respond instantly and manage turn-taking, backchanneling, and interruptions, but tend to produce semantically weaker replies. The method aims to combine immediate responsiveness with improved response content.