Neural Information Processing Systems http://nips.cc/

We focus on a stochastic learning model where the learner observes a finite set of training examples and the output of the learning process is a data-dependent distribution over a space of hypotheses.

This provides an alternative explanation to the Bayesian Occam's razor criteria, under the assumption that the data

Neural Information Processing Systems http://nips.cc/

The Probably Approximately Correct (PAC) Bayes framework (McAllester, 1999) can incorporate knowledge about the learning algorithm and (data) distribution through the use of distribution-dependent priors, yielding tighter generalization bounds on data-dependent posteriors. Using this flexibility, however, is difficult, especially when the data distribution is presumed to be unknown. We show how an e-differentially private data-dependent prior yields a valid PAC-Bayes bound, and then show how non-private mechanisms for choosing priors can also yield generalization bounds. As an application of this result, we show that a Gaussian prior mean chosen via stochastic gradient Langevin dynamics (SGLD; Welling and Teh, 2011) leads to a valid PAC-Bayes bound given control of the 2-Wasserstein distance to an e-differentially private stationary distribution. We study our datadependent bounds empirically, and show that they can be nonvacuous even when other distribution-dependent bounds are vacuous.

The Probably Approximately Correct (PAC) Bayes framework (McAllester, 1999) can incorporate knowledge about the learning algorithm and (data) distribution through the use of distribution-dependent priors, yielding tighter generalization bounds on data-dependent posteriors. Using this flexibility, however, is difficult, especially when the data distribution is presumed to be unknown. We show how a differentially private data-dependent prior yields a valid PAC-Bayes bound, and then show how non-private mechanisms for choosing priors can also yield generalization bounds. As an application of this result, we show that a Gaussian prior mean chosen via stochastic gradient Langevin dynamics (SGLD; Welling and Teh, 2011) leads to a valid PAC-Bayes bound due to control of the 2-Wasserstein distance to a differentially private stationary distribution. We study our data-dependent bounds empirically, and show that they can be nonvacuous even when other distribution-dependent bounds are vacuous.

A year later, Barrick has parted ways with its chief innovation officer, chief digital officer and many of the team tasked with making this transformation a reality, according to people familiar with the matter. The revolution in machine learning, as predicted by Barrick Chairman John Thornton and other mining executives, has yet to come. Miners have said digital technologies like artificial intelligence, or AI, will revolutionize one of the world's oldest industries in the same way it has changed other businesses, from retail to hailing a cab. Some experts say the promise of AI in mining has been overhyped and progress has been slow. Companies, including Barrick and giants such as Rio Tinto PLC and BHP Group Ltd., are running some AI-led projects. But implementation at some companies has hit cultural hurdles.

As the thirst for technology increases and the demand for smartphones, electric cars and other complex technological devices grows, the amount of resources, minerals and metals we need will only increase, but can we meet this demand? There are few sectors and industries that have not been impacted by technological advancement. Whether it's improving efficiency, enhancing transparency or transforming the supply chain, big data, machine learning and AI are poised to reshape the mining sector as we know it; and the timing couldn't be any better. At the recent Big Data and AI conference held in Toronto, the topic of mining disruption through technology was front and center. Speaker Denis Laviolette, president and CEO of GoldSpot Discovery, highlighted the need for the mining sector to not only embrace the recent advancements, but to also quickly look for ways to integrate these innovations into its business model.

The Probably Approximately Correct (PAC) Bayes framework (McAllester, 1999) can incorporate knowledge about the learning algorithm and data distribution through the use of distribution-dependent priors, yielding tighter generalization bounds on data-dependent posteriors. Using this flexibility, however, is difficult, especially when the data distribution is presumed to be unknown. We show how an {\epsilon}-differentially private data-dependent prior yields a valid PAC-Bayes bound, and then show how non-private mechanisms for choosing priors obtain the same generalization bound provided they converge weakly to the private mechanism. As an application of this result, we show that a Gaussian prior mean chosen via stochastic gradient Langevin dynamics (SGLD; Welling and Teh, 2011) leads to a valid PAC-Bayes bound, despite SGLD only converging weakly to an {\epsilon}-differentially private mechanism. As the bounds are data-dependent, we study the bounds empirically on synthetic data and standard neural network benchmarks in order to illustrate the gains of data-dependent priors over existing distribution-dependent PAC-Bayes bound.

We exhibit a strong link between frequentist PAC-Bayesian bounds and the Bayesian marginal likelihood. That is, for the negative log-likelihood loss function, we show that the minimization of PAC-Bayesian generalization bounds maximizes the Bayesian marginal likelihood. This provides an alternative explanation to the Bayesian Occam's razor criteria, under the assumption that the data is generated by an i.i.d. distribution. Moreover, as the negative log-likelihood is an unbounded loss function, we motivate and propose a PAC-Bayesian theorem tailored for the sub-gamma loss family, and we show that our approach is sound on classical Bayesian linear regression tasks.