DA increases the size of the training data set, generally, in two ways. The first approach expands thetraining data byperforming expert-defined content preserving transformations toexisting data points [2].

For example, the information bottleneck method seeks a good generalization by finding a minimal description of the input that is maximally informative about the label variable, where minimality and informativeness are both measuredbyShannon'smutualinformation.

For example, the information bottleneck method seeks a good generalization by finding a minimal description of the input that is maximally informative about the label variable, where minimality and informativeness are both measuredbyShannon'smutualinformation.

Adversarial training is a popular method to give neural nets robustness against adversarial perturbations. In practice adversarial training leads to low robust trainingloss.

Much ofreinforcement learning theory isbuiltontopoforacles that arecomputationally hard to implement.