Building a Video Search Engine

A natural progression in the field of computer vision following unprecedented progress in image classification tasks is towards video and video understanding, especially how it relates to identifying human subjects and activities. A number of datasets and benchmarks are being established in this area¹. In parallel, further progress is being made in 2D image related computer vision tasks such as fine-grained classification, image segmentation, 3D image construction, robot vision, scene flow estimation and human pose estimation. As part of my final Data Science project at Metis bootcamp, I've decided to marry these two parallel tracks -- video and human pose estimation in specific -- to create a content-based video search engine. Since applying 2D human pose estimation for video search is a novel idea with "no proof of concept", I have simplified my approach by selecting a single performer, fixed location single camera video footage of Salsa dance videos.