Deep learning (DL) systems are notoriously difficult to test and debug due to the lack of correctness proof and the huge test input space to cover. Given the ubiquitous unlabeled test data and high labeling cost, in this paper, we propose a novel test prioritization technique, namely TestRank, which aims at revealing more model failures with less labeling effort. TestRank brings order into the unlabeled test data according to their likelihood of being a failure, i.e., their failure-revealing capabilities.

Deep learning (DL) systems are notoriously difficult to test and debug due to the lack of correctness proof and the huge test input space to cover. Given the ubiquitous unlabeled test data and high labeling cost, in this paper, we propose a novel test prioritization technique, namely TestRank, which aims at revealing more model failures with less labeling effort. TestRank brings order into the unlabeled test data according to their likelihood of being a failure, i.e., their failure-revealing capabilities. To be specific, we first build a similarity graph on both unlabeled test samples and labeled samples (e.g., training or previously labeled test samples). Then, we conduct graph-based semi-supervised learning to extract contextual features from the correctness of similar labeled samples.

For the many companies that rely on pricing as a competitive advantage, they need to start evaluating AI and machine learning on their IT platform roadmaps now. Staying at competitive parity and turning AI- and machine learning-based expertise into a pricing and revenue management strength needs to be a priority. Data is a proven panacea for fear, and given the new market dynamics many companies are facing, it's the most reliable way to make decisions. Harnessing Pricing Power to Create Lasting Value, Bain & Company, February 24, 2020. Harnessing Pricing Power to Create Lasting Value, Bain & Company, February 24, 2020.

This video course will demonstrate the steps for analyzing unstructured data with the R/R Studio software. The approaches will be illustrated using practical applications for business, healthcare, and retail data, among others. At the end the video course you will have mastered obtaining and visualizing data with R. You will also be confident with data cleaning, preparation, and sentiment analysis with R. Dr. Bharatendra Rai is a professor of Business Statistics and Operations Management in the Charlton College of Business at UMass Dartmouth. He received his Ph.D. in Industrial Engineering from Wayne State University, Detroit.