Review for NeurIPS paper: FrugalML: How to use ML Prediction APIs more accurately and cheaply

Each API has some predictive accuracy and quality score (confidence) but also has an assigned cost, which we'd like to minimize. The authors give a method to accomplish this: a base API is chosen based on learnt conditional accuracies which might be overruled by an add-on API if the quality score is not sufficiently high. The optimal strategy is generated via solving a stated optimization problem. The paper presents some neat experiments with this method on computer vision and NLP datasets with real-world APIs. These appear promising in that the generated strategy reduces costs while still achieving high predictive accuracies.