Research Summary
A gesture could be defined as unit of creation or perception of movement that may or may not communicative.
A fundamental question in gesture analysis and pattern recognition involving gestures is "How do we effectively model gestures, to recognize (i.e. tag) them?" The answer to this question has been anything but obvious. Present approaches to gesture analysis have been influenced by the work in the field of computational linguistics wherein many aspects of languages (such as the segmentation of continuous speech into discrete words and recognition of those words) are well understood by linguists and philosophers. This knowledge has been embodied in algorithms that are used as the basis for practical speech recognition systems, text-to-speech synthesizers, automated voice response systems, web search engines, text editors, and parsers.
Methods for gesture analysis have much in common with computational linguistics, and gesture analysis researchers have employed algorithms (such as HMM), that were specifically developed to solve computational linguistic problems. However conventional method of modeling gestures as a series of poses does not allow development of generic gesture recognition and segmentation models as the number of poses the human body can assume is very large.
I have developed gesture recognition and segmentation models that model gestures as a series of activities in the human body hierarchy. For segmentation, a gesture is modeled as a series of activities in the human body hierarchy. The model accounts for mannerism gestures by developing profiles of individuals and how they perform gestures. The profile guides the gesture segmentation engine on what movement blocks constitute a gesture in a given motion sequence. Figure 1 shows the human body hierarchy
Figure 1. Human Body Hierarchy
For gesture recognition, the gesture is modeled as a series of activities in the human body segments and joints. A Body Distance Coupled Hidden Markov Model based on distances between joints and segments captures the gestures and represents them. This model has been used to recognize a library of over 230 gestures with 90.2% accuracy. The main contribution of this work is to develop models wherein every gesture can be represented by the same HMM. This modeling of gestures may lead to standardization of the human motion HMM just as phonemes did for the speech recognition HMM's. Figure 2 shows the proposed formulation
Figure 2. Gesture Recognition Module
The following are links to my gesture segmentation, gesture recognition work. I have also completed a gesture annotation software which is enlisted.
ASU Main Website