First-order optimization methods are a foundational tool in machine learning.

First-order optimization methods are a foundational tool in machine learning.

Bayesian inference allows expressing the uncertainty of posterior belief under a probabilistic model given prior information and the likelihood of the evidence. Predominantly, the likelihood function is only implicitly established by a simulator posing the need for simulation-based inference (SBI).

This blog post is based on the work BaNEL: Exploration Posteriors for Generative Modeling Using Only Negative Rewards . The ultimate aim of machine learning research is to push machines beyond human limits in critical applications, including the next generation of theorem proving, algorithmic problem solving, and drug discovery. A standard recipe involves: (1) pre-training models on existing data to obtain base models, and then (2) post-training them using scalar reward signals that measure the quality or correctness of the generated samples. The probability of producing a positive-reward sample can be so low that the model may go through most of the training without ever encountering a positive reward. Calls to the reward oracle can be expensive or risky, requiring costly simulations, computations, or even physical experiments.

First-order optimization methods are a foundational tool in machine learning.