DCTD: Deep Conditional Target Densities for Accurate Regression

While deep learning-based classification is generally addressed using standardized approaches, a wide variety of techniques are employed for regression. In computer vision, one particularly popular such technique is that of confidence-based regression, which entails predicting a confidence value for each input-target pair (x,y). While this approach has demonstrated impressive results, it requires important task-dependent design choices, and the predicted confidences often lack a natural probabilistic meaning. We address these issues by proposing Deep Conditional Target Densities (DCTD), a novel and general regression method with a clear probabilistic interpretation. DCTD models the conditional target density p (y x) by using a neural network to directly predict the un-normalized density from (x,y). This model of p (y x) is trained by minimizing the associated negative log-likelihood, approximated using Monte Carlo sampling. We perform comprehensive experiments on four computer vision regression tasks. Our approach outperforms direct regression, as well as other probabilistic and confidence-based methods. Notably, our regression model achieves a 1.9% AP improvement over Faster-RCNN for object detection on the COCO dataset, and sets a new state-of- the-art on visual tracking when applied for bounding box regression. Supervised regression entails learning a model capable of predicting a continuous target value y from an input x, given a set of paired training examples. It is a fundamental machine learning problem with many important applications within computer vision and other domains. While all of these tasks benefit from accurate regression of the target values, high accuracy can even be safety-critical in e.g. automotive and medical applications. Today, such regression problems are commonly tackled using Deep Neural Networks (DNNs), due to their ability to learn powerful feature representations from data. While classification is generally addressed using standardized losses and output representations, a wide variety of different techniques are employed for regression. The most conventional strategy is to train a DNN to directly predict a target y given an input x (Lathuili ere et al., 2019). The training data { (x i,y i)} 2000 i 1 is generated by the ground truth conditional target density p (y x) . DCTD models p (y x) by directly predicting the un-normalized density from the input-target pair (x,y), and is trained by minimizing the associated negative log-likelihood.