Generative Modeling for Small-Data Object Detection

This paper explores object detection in the small data regime, where only a limited number of annotated bounding boxes are available due to data rarity and annotation expense. This is a common challenge today with machine learning being applied to many new tasks where obtaining training data is more challenging, e.g. in medical images with rare diseases that doctors sometimes only see once in their lifetime. In this work we explore this problem from a generative modeling perspective by learning to generate new images with associated bounding boxes, and using these for training an object detector . W e show that simply training previously proposed generative models does not yield satisfactory performance due to them optimizing for image realism rather than object detection accuracy. T o this end we develop a new model with a novel unrolling mechanism that jointly optimizes the generative model and a detector such that the generated images improve the performance of the detector . W e show this method outperforms the state of the art on two challenging datasets, disease detection and small data pedestrian detection, improving the average precision on NIH Chest X-ray by a relative 20% and localization accuracy by a relative 50%. 1. Introduction Generative Adversarial Networks (GANs) [6] have recently advanced significantly, with the latest models [3, 12] being able to generate high quality photo-realistic images that are almost indistinguishable from real images. A natural question that has recently started being explored [17, 24, 26] is whether these generated images are useful in some other ways; for example, could they be useful training data for downstream tasks? One common computer vision task that could benefit from generated data is object detection [21, 25] which currently requires a large amount of training data to obtain good performance. But for many object detection tasks, This work was conducted when Lanlan Liu was an intern at Google.