Transfer learning for conflict and duplicate detection in software requirement pairs

The conflict and duplicate requirement detection problem is formulated as a requirement pair classification task. A novel architecture, SR-BERT, is proposed to detect the conflicts and duplicate requirement pairs. The capabilities of sequential and cross-domain transfer learning models are assessed for the requirement pair classification tasks. Rule-based filtering techniques are explored to validate the cross-domain model classifications. Abstract Consistent and holistic expression of software requirements is important for the success of software projects. In this study, we aim to enhance the efficiency of the software development processes by automatically identifying conflicting and duplicate software requirement specifications. We formulate the conflict and duplicate detection problem as a requirement pair classification task. We design a novel transformers-based architecture, SR-BERT, which incorporates Sentence-BERT and Bi-encoders for the conflict and duplicate identification task. Furthermore, we apply supervised multi-stage fine-tuning to the pre-trained transformer models. We test the performance of different transfer models using four different datasets. We find that sequentially trained and fine-tuned transformer models perform well across the datasets with SR-BERT achieving the best performance for larger datasets. We also explore the cross-domain performance of conflict detection models and adopt a rulebased filtering approach to validate the model classifications. Our analysis indicates that the sentence pair classification approach and the proposed transformer-based natural language processing strategies can contribute significantly to achieving automation in conflict and duplicate detection. Corresponding author Email address: mcevik@ryerson.ca Preprint submitted to Journal of Systems & Software January 11, 2023 1. Introduction Software requirements outline the characteristics, functionalities, design and implementation constraints of a software for the development team. These requirements are expected to be accurate, consistent and comprehensive. Consistency of the requirements can be particularly important for the success of the software development project since contradictory and redundant requirements directly impact the project delivery time and associated costs.