We thank the reviewers for their valuable feedback, which will improve the paper. Regarding the reviewer's comments about applications, we chose to limit the number of applications to three because Cauchy, which has unbounded variance), in contrast to our mechanisms. As requested, we will add a discussion about related work on lower bounds for private mechanisms. For the reviewer's main comment on the contributions of this paper with regard to Asi & Duchi 2020, we believe Such general (vector-valued) functions are the main focus of this submission. We thank the reviewer for bringing our attention to the Reimherr & A wan's K-norm mechanism (2019), which certainly We will discuss this work more carefully in the final version.

We graciously thank the reviewers for their helpful comments. We clarify some details of the article below. In fact, this article shows that even if FE isn't as mature as homomorphic We do detail and reference many notions from cryptology. ML community may not be familiar with those new concepts, and we sought to introduce them carefully and rigorously. In return, classical notions of ML do not need to be referenced as much because they are well established.

We thank the reviewers for their insightful feedback. While we realize this may be subjective, even in this "oracle" setting We also notice similar patterns as Table 2 in the paper, e.g. this phenomenon is much more present in Dec-Dec attention, whereas Enc-Dec attention suffers much more from keeping only one head (-18.89 "only one head is sufficient" claim in the abstract, introduction and conclusion. We see two issues here. We will add this to the revised paper.

We thank the reviewers for their detailed reviews and constructive feedback. It is not known how tight any of these bounds are. We will clarify this point in the final version. Red lines are GD while blue lines are NGD (Hessian-free). Solid lines are training curves while dashed lines are testing curves.