Learning from failure to tackle extremely hard problems

This blog post is based on the work BaNEL: Exploration Posteriors for Generative Modeling Using Only Negative Rewards . The ultimate aim of machine learning research is to push machines beyond human limits in critical applications, including the next generation of theorem proving, algorithmic problem solving, and drug discovery. A standard recipe involves: (1) pre-training models on existing data to obtain base models, and then (2) post-training them using scalar reward signals that measure the quality or correctness of the generated samples. The probability of producing a positive-reward sample can be so low that the model may go through most of the training without ever encountering a positive reward. Calls to the reward oracle can be expensive or risky, requiring costly simulations, computations, or even physical experiments.