This paper presents an auditing procedure for the Differentially Private Stochastic Gradient Descent (DP-SGD) algorithm in the black-box threat model that is substantially tighter than prior work. The main intuition is to craft worst-case initial model parameters, as DP-SGD's privacy analysis is agnostic to the choice of the initial model parameters. For models trained on MNIST and CIFAR-10 at theoretical ε = 10.0, our auditing procedure yields empirical estimates of ε

The distributionally robust tree structured prediction problem based on moment divergence in Eq. (1) can be rewritten as min E The objective function is convex in P and concave in Q because it is affine in both. The simplex constraints are omitted. Theorem 2. Given m samples, a non-negative loss l(,) such that |l(,)| K, a feature function ϕ(,) such that ϕ(,) B, a positive ambiguity level ε > 0, then, for any ρ (0, 1], with a probability at least 1 ρ, the following excess true worst-case risk bound holds: () ln(4/ρ) max As per the assumption, ϕ(,) B. This further implies that f(θ K ε i = 1, 2. ε We then follow the proof of Theorem 3 in Farnia and Tse [2016]. Our formulation differs from Nowak-Vila et al. [2020] in the fact that we allow probabilistic prediction to be ground truth. Proposition 4. Let G be a multi-graph.

Learning to cooperate with friends and compete with foes is a key component of multi-agent reinforcement learning. Typically to do so, one requires access to either a model of or interaction with the other agent(s). Here we show how to learn effective strategies for cooperation and competition in an asymmetric information game with no such model or interaction. Our approach is to encourage an agent to reveal or hide their intentions using an information-theoretic regularizer. We consider both the mutual information between goal and action given state, as well as the mutual information between goal and state. We show how to optimize these regularizers in a way that is easy to integrate with policy gradient reinforcement learning. Finally, we demonstrate that cooperative (competitive) policies learned with our approach lead to more (less) reward for a second agent in two simple asymmetric information games.

This paper presents an auditing procedure for the Differentially Private Stochastic Gradient Descent (DP-SGD) algorithm in the black-box threat model that is substantially tighter than prior work. The main intuition is to craft worst-case initial model parameters, as DP-SGD's privacy analysis is agnostic to the choice of the initial model parameters. For models trained on MNIST and CIFAR-10 at theoretical ε = 10.0, our auditing procedure yields empirical estimates of ε

While sequential decision-making environments often involve high-dimensional observations, not all features of these observations are relevant for control. In particular, the observation space may capture factors of the environment which are not controllable by the agent, but which add complexity to the observation space. The need to ignore these "noise" features in order to operate in a tractably-small state space poses a challenge for efficient policy learning. Due to the abundance of video data available in many such environments, task-independent representation learning from action-free offline data offers an attractive solution. However, recent work has highlighted theoretical limitations in action-free learning under the Exogenous Block MDP (Ex-BMDP) model, where temporally-correlated noise features are present in the observations. To address these limitations, we identify a realistic setting where representation learning in Ex-BMDPs becomes tractable: when action-free video data from multiple agents with differing policies are available. Concretely, this paper introduces CRAFT (Comparison-based Representations from Action-Free Trajectories), a sample-efficient algorithm leveraging differences in controllable feature dynamics across agents to learn representations. We provide theoretical guarantees for CRAFT's performance and demonstrate its feasibility on a toy example, offering a foundation for practical methods in similar settings.

This appendix includes the missing proofs of the results presented in the main text, additional results, and some helpful lemmas. Let P and Q be two probability measures defined on the same measurable space (Ω, F), such that P is absolutely continuous with respect to Q. Then the Donsker-Varadhan dual characterization of Kullback-Leibler divergence states that { [ Lemma 3. Let X and Y be independent random variables. If g is a measurable function such that g(x, Y) is σ-subgaussian and E g(x, Y) = 0 for all x X, then g(X, Y) is also σ-subgaussian. To prove the last part of the lemma, we just use the Markov's inequality and combine with this last result: ɛ = P Ψ) E Taking expectation over u on both sides, then swapping the order between expectation over u and absolute value (using Jensen's inequality), we get [ Furthermore, each of these summands has zero mean. Taking expectation over u on both sides and using Jensen's inequality to switch the order of absolute value and expectation of u, we get [ Taking expectation over z on both sides, and then using Jensen's inequality to switch the order of absolute value and expectation of z, we get [ Hence, it can be bounded by 1/(4n).

We derive information-theoretic generalization bounds for supervised learning algorithms based on the information contained in predictions rather than in the output of the training algorithm. These bounds improve over the existing informationtheoretic bounds, are applicable to a wider range of algorithms, and solve two key challenges: (a) they give meaningful results for deterministic algorithms and (b) they are significantly easier to estimate. We show experimentally that the proposed bounds closely follow the generalization gap in practical scenarios for deep learning.

Correspondence should be addressed to: qiang.cheng@uky.edu. The architecture of our algorithm is shown in Figure 1. Our feature scorer and selector are not just the direct definitions for the importance of features; instead, the feature scorer and selector will be estimated iteratively through the NN and sub-NN. For k, it is subject to the practical problem, which is somewhat similar to the number of clusters in k-means clustering. For the training based on Eq. (2) of the main text, in each iteration of backpropagation, The selector will require the gradients of the features having the top-k weights in magnitude for this iteration while having no effect on the gradients of other features.