Dynamic Sampling Of Video To Imagery For Deep Learning

While today's deep learning systems are able to natively analyze video, the large file sizes of high resolution movies present unique challenges in terms of storage space and computational requirements. Sampling them into sequences of still images not only allows for real-time processing of unlimited-length videos but opens the door for creative new applications like "video ngrams." The most straightforward way to sample a video into a sequence of still images is to use a fixed-rate time-based mechanism such as one frame per second. This kind of sampling is supported natively by most tools like ffmpeg and provides a simplistic and robust workflow. At the same time, it is highly inefficient, especially for videos where there is a lot of repetition. In the case of television news, a considerable portion of the airtime is devoted to motionless anchors sitting in an unchanging studio, meaning there can be quite literally thousands of nearly identical frames in a single broadcast.