Review for NeurIPS paper: Pre-training via Paraphrasing

Weaknesses: The overall idea of retrieving related texts for pre-training is similar to REALM. The adopted retrieval method needs more refinement in detail. It is roughly based on overall document cosine similarity, which may involve much noise. Besides, the retrieval task is latently problematic, as it is closely related to the training target of the model and is not capable of reflecting the effectiveness of the encoder. The machine translation also does not measure the performance of the encoder because it relies on the decoder to generate the correct target sequence. The experiments and experimental settings are insufficient.