Out-of-domain Detection for Natural Language Understanding in Dialog Systems

In natural language understanding components, detecting out-of-domain (OOD) inputs is important for dialogue systems since wrongly accepting these OOD utterances that are not currently supported may lead to catastrophic failures of the entire system. Entropy regularization is an effective solution to avoid such failures, however, its computation heavily depends on OOD data, which are expensive to collect. In this paper, we propose a novel text generation model to produce high-quality OOD samples and thereby improve the performance of OOD detection. The proposed model can also utilize a set of unlabeled data to improve the effectiveness of these generated OOD samples. Experiments show that our method can effectively improve the OOD detection performance of a NLU module. 1 Introduction Natural Language Understanding (NLU) in dialog systems, particularly including task-oriented dialog systems and intelligent personal assistants, is vital for understanding users' inputs and making effective interactions. NLU maps text inputs to structured user intents, and decides the downstream processing pipelines of a dialog system, thereby becoming a precursor for the success of such systems. Recently, various deep neural network (DNN) based NLU models have been proposed and applied in real-world applications (Kim et al., 2018; Sarikaya, 2017; Y oo et al., 2018). Most existing DNN based NLU modules are built by following a closed-world assumption (Fei and Liu, 2016), i.e, the data used in the training and test phrase are drawn from the same distribution. However, such an assumption is commonly violated in practical systems that are deployed in a dynamic or open environment. Specifically, practical NLU systems often encounter o ut-o f-d omain (OOD) inputs that are not supported by the system and thus not observed in the training data.