Detecting dialogue breakdown in real time is critical for conversational AI systems, because it enables taking corrective action to successfully complete a task. In spoken dialog systems, this breakdown can be caused by a variety of unexpected situations including high levels of background noise, causing STT mistranscriptions, or unexpected user flows. In particular, industry settings like healthcare, require high precision and high flexibility to navigate differently based on the conversation history and dialogue states. This makes it both more challenging and more critical to accurately detect dialog breakdown. To accurately detect breakdown, we found it requires processing audio inputs along with downstream NLP model inferences on transcribed text in real time. In this paper, we introduce a Multimodal Contextual Dialogue Breakdown (MultConDB) model. This model significantly outperforms other known best models by achieving an F1 of 69.27.

This paper describes the dialog robot system designed by Team Irisapu for the preliminary round of the Dialogue Robot Competition 2023 (DRC2023). In order to generate dialogue responses flexibly while adhering to predetermined scenarios, we attempted to generate dialogue response sentences using OpenAI's GPT-3. We aimed to create a system that can appropriately respond to users by dividing the dialogue scenario into five sub-scenarios, and creating prompts for each sub-scenario. Also, we incorporated a recovery strategy that can handle dialogue breakdowns flexibly. Our research group has been working on research related to dialogue breakdown detection, and we incorporated our findings to date in this competition. As a result of the preliminary round, a bug in our system affected the outcome and we were not able to achieve a satisfactory result. However, in the evaluation category of "reliability of provided information", we ranked third among all teams.

Building user trust in dialogue agents requires smooth and consistent dialogue exchanges. However, agents can easily lose conversational context and generate irrelevant utterances. These situations are called dialogue breakdown, where agent utterances prevent users from continuing the conversation. Building systems to detect dialogue breakdown allows agents to recover appropriately or avoid breakdown entirely. In this paper we investigate the use of semi-supervised learning methods to improve dialogue breakdown detection, including continued pre-training on the Reddit dataset and a manifold-based data augmentation method. We demonstrate the effectiveness of these methods on the Dialogue Breakdown Detection Challenge (DBDC) English shared task. Our submissions to the 2020 DBDC5 shared task place first, beating baselines and other submissions by over 12\% accuracy. In ablations on DBDC4 data from 2019, our semi-supervised learning methods improve the performance of a baseline BERT model by 2\% accuracy. These methods are applicable generally to any dialogue task and provide a simple way to improve model performance.