Accurate online action and gesture recognition system using detectors and Deep SPD Siamese Networks

Human activity recognition is an important research topic in pattern recognition field. It has been the subject of many studies in the past two decades because of its importance in numerous areas such as security, health, daily activity, energy consumption and robotics. Recently, some works on the recognition of hand gestures or human actions from skeletal data are based on the modeling of the skeleton's movement as manifold-based representation and proposed deep neural networks on this structure [1, 2, 3]. These approaches demonstrated their potential in the processing of skeletal data. Most of them are applied on offline human action recognition which is useful in time-limited tasks. However, in many applications, simply recognizing a single gesture in a given segmented sequence is not enough, especially in monitoring systems and virtual-reality devices which need to detect human movements moment by moment in continuous videos. In these online recognition systems, it is important to detect the existence of an action as early as possible after its beginning. It is also essential to determine the nature of the movement within a sequence of frames, without having information about the number of gestures present within the video, their starting times or their durations, unlike the segmented action recognition. In this paper, we propose to use a manifold-based model in order to build an online motion recognition system that detects and identifies different human activities in unsegmented skeletal sequences.