A Additional Discussions

In this work, we focus on the optimal membership inference adversary. The following lemmas are used in the proofs of Theorems 3.2 and D.1. We start with the first term. In the last line, we use the same argument as in Section 2.2 of [ Adding this with the result for the first term gives the desired result. Thus, the regressor memorizes the training data and the training error is equal to zero.