Neural Information Processing Systems http://nips.cc/

Modularity is a compelling solution to continual learning (CL), the problem of modeling sequences of related tasks.

Statistical learning theory is the foundation of machine learning, providing theoretical bounds for the risk of models learned from a (single) training set, assumed to issue from an unknown probability distribution.

Despite its popularity, self-training is well-believed to be unreliable and often leads to training instability.

Based on this observation, we adopt ageneralized version ofthe JensenShannon divergence for multiple distributions to encourage consistency around data points. Using this loss function, we show state-of-the-art results on both synthetic(CIFAR),andreal-world(e.g.WebVision)noisewithvaryingnoiserates.

It is not clear, however, whether these theoretical results apply to actual distributions such as images.

The ability of machine learning to generate structured objects has progressed dramatically in the lastfewyears.