We describe a real-time computer vision and machine learning sys(cid:173) tem for modeling and recognizing human actions and interactions. Two different domains are explored: recognition of two-handed motions in the martial art'Tai Chi', and multiple- person interac(cid:173) tions in a visual surveillance task. Our system combines top-down with bottom-up information using a feedback loop, and is formu(cid:173) lated with a Bayesian framework. Two different graphical models (HMMs and Coupled HMMs) are used for modeling both individual actions and multiple-agent interactions, and CHMMs are shown to work more efficiently and accurately for a given amount of train(cid:173) ing. Finally, to overcome the limited amounts of training data, we demonstrate that'synthetic agents' (Alife-style agents) can be used to develop flexible prior models of the person-to-person inter(cid:173) actions.

We propose a supervised approach incorporating groupfeature sparsity in multi-class kernel logistic regression(GFR-MKLR). The need for group sparsity arises inseveral practical situations where a subset of a set offactors can explain a predicted variable and each factorconsists of a group of variables. We apply our approachfor predicting human interactions based on bodyparts motion (e.g., hands, legs, head, etc.) where imagefeatures are organised in groups corresponding to bodyparts. Our approach, leads to sparse models by assigningweights to groups of features having the highest discriminationbetween different types of interactions. Experimentsconducted on the UT-Interaction dataset havedemonstrated the performance of our method with regardto stat-of-art methods.