Appendix of Learning to Break the Loop Analyzing and Mitigating Repetitions for Neural Text Generation

Previous work [2, 1] has observed that standard training and greedy decoding usually cause models to generate consecutive repetitive texts. These consecutive repetitive texts are redundant and do not convey new information, which is avoided in human language. There are three types of consecutive repetitions: word-level, phrase-level and sentence-level. The phrase-level means that a phrase consisting of several words is repeated consecutively. The sentence in our paper refers to a sequence split by '.!?' is repeated consecutively 2. We calculate the ratio of consecutive repetition in a sequence x as follows.