Flatness of the loss curve is conjectured to be connected to the generalization ability of machine learning models, in particular neural networks.

In Section A.7, we will show that super-exponential scaling Because the student's decision boundary is invariant to an overall scaling of N (24) which represents the overlap between each replicated student and the probe student. H -function in Eq. 63 becomes increasingly sharp, approaching a step function: H null t null null q ρ Finally, Eq. 76 reveals that as we prune more aggressively the information gain per example Which allows us to produce to trace the Pareto frontier in Figure 1 F . What happens if the probe student does not perfectly match the teacher? Hence the data ultimately stops concentrating around the teacher's decision boundary, and the information gained from each new example goes to zero. The saddle point equations Eq. 66,67 reveal that the optimal pruning policy varies as a function of The dashed purple line indicates the "keep easy" frontier (computed using In this section we investigate this question.

In this paper, we study a new model consisting of a general convex or nonconvex objectives and a variety of continuous nonconvex sparsity-inducing constraints.

ReLU networks, showing that its fixed points can have higher ranks.

ReLU networks, showing that its fixed points can have higher ranks.

The use of a global population can lead to potentially misleading results when local model behaviour is of interest. Hence we consider the formulation of neighbourhood reference distributions that improve the local interpretability of Shapley values.

With modern high-density probes (Jun et al., 2017), neuroscientists can observe the spiking activity