Current methods like membership inference or active IP protection either work only in settings where the suspected text is known or do not provide reliable statistical guarantees. We discover that, on the contrary, it is possible to reliably determine if a language model was trained on synthetic data if that data is output by a watermarked LLM.

In this paper, the authors define a list of properties (they refer to them as axioms).

Accuracy that was not previously used in classification literature.

The kernel Maximum Mean Discrepancy (MMD) is a popular multivariate distance metric between distributions that has found utility in two-sample testing.

Table 1 present the evaluation results. We also evaluate our method on the CIFAR benchmark with various model architectures. The best three results are highlighted with red, blue, and cyan. The best two results are highlighted with red and blue . Table 2 compares the performance against all the post hoc baselines.