Parameter-Efficient Low-Resource Dialogue State Tracking by Prompt Tuning

The computing and data resource-hungry Dialogue state tracking (DST) that extracts structured issues are more severe in the real-world deployment conversation progress in a list of slot-value where LMs tuned for different domains and pairs from unstructured dialogue utterances is an essential tasks need to be trained and hosted, and a typical component of a dialogue system (Wang and dialogue system has to serve dozens of such LMs Lemon, 2013). Unlike classification-based models (Maronikolakis and Schütze, 2021; Strubell et al., that pick the slot value from given candidate (Ye 2019; Lacoste et al., 2019). This leads to a high cost et al., 2021; Chen et al., 2020), recent works formulate of the development and service of dialogue systems DST as a conditional generation task (Gao and constrains offline deployment. In addition, limited et al., 2019; Lin et al., 2020), where the concatenation data is available for a new domain or task. of dialogue history and a slot-specific prompt We propose a parameter-efficient and dataefficient are fed to generative models and the text generation DST model for low-resource settings, output are decoded to predicted slot values (Ham which only needs to update 0.08% of parameters et al., 2020; Hosseini-Asl et al., 2020). This formulation compared with the previous best model, by enjoys the benefit of generalizability to keeping LM parameters frozen and introducing unseen domains and slot types beyond a defined dialogue soft prompt tokens to represent task properties ontology (Li et al., 2021; Peng et al., 2021). of different slots. Figure 1 gives an overview of General prompting methods use a textual prompt our model. The only prior work we are aware of to provide task information to the LM (Liu et al., that only updates prompt token embeddings and 2021; Ma et al., 2023b). Prior works have variations thus parameter-efficient is Zhu et al. (2022), but that update different parameter combinations it focuses on continual domain adaptation and with such as both LM and prompt token embeddings a significant amount of training data. Work done while at Amazon.