With direct access to human-written reference as memory, retrieval-augmented generation has achieved much progress in a wide range of text generation tasks.

However,duetogrowing dataset sizes and stricter privacy regulations, it is increasingly common not to have access to the data that was used to train the teacher. We propose a novel method which trains a student to match the predictions of its teacher without using anydata ormetadata. Weachievethisbytraining anadversarial generator to search for images on which the student poorly matches the teacher, and then using them to train the student.

Neural Information Processing Systems http://nips.cc/

Neural Information Processing Systems http://nips.cc/

With acareful study, we find amore general problem which is rooted in low-frequency words in the text corpus.

Neural Information Processing Systems http://nips.cc/