Research
Synopsis Research Summary
The desire for natural and intuitive human machine interaction
has led to the inclusion of haptics in such I/O interfaces. The user is
able to control inputs to the system though hand movements and in turn receives
feedback through vibro-tactile stimulation in the hands. This allows perception
of distal/virtual environments (for my research approach on how to perceive
distal environments, please see the visio-haptic section). While the potential
of haptics in natural human machine interaction is undisputable, the realization
of such means is still a long way ahead. This is especially true, when designing
haptic systems wherein haptics is the primary sensory perception channel
(for example consider haptic systems for individuals who are blind or haptic
systems for telesurgery and teleoperation).
There are considerable research challenges to development of ‘natural’
haptic interfaces. One of the major impediments is that the human tactile
system is not as well understood as other modalities such as vision, speech
or motor systems. The study of human tactile abilities is a recent endeavor
and many of the available systems still do not incorporate the domain knowledge
of psychophysics, biomechanics and neuromechanical elements of haptic perception.
Development of smart and effective haptic interfaces and devices requires
extensive studies that link perceptual phenomena with bio-mechanical measures
and incorporation of such domain knowledge in the engineering of haptic
interfaces. In my research I propose to develop a generic methodology
for design of haptic systems where haptics is the primary sensory channel
for perception.
Note: I work with haptic gloves CyberTouch® and this discussion
is presented with the configuration of the gloves in mind. However the discussion
does apply to other systems.
Research Approach
The key points of conceptual framework that guides my approach
to development of haptic user interfaces are
(1) Incorporation of physiological
and psychological factors of haptic perception in the design of haptic user
interfaces. This may account to encoding present knowledge
base on haptic perception in the applications or further augmenting the
knowledge base by performing physiological/psychological experiments.
(2) Design and develop novel methods of haptic
rendering that achieve the goal of virtual/distal environment perception.
It is important to realize that realistic haptic rendering (which I define
as rendering that aims to mimic the haptic features of real environments)
while being most desirable, is not an easy task. The state of the art haptic
devices have a limited force and tactile feedback devices that do not mimic
the scale of the human tactile system. This severely limits the ability
of computerized applications to provide realistic feedback. Innovative approaches
that are based on psychological findings (as I will describe later) can
render objects without completely mimicking their features in real environments
and yet achieve near complete distal environment perception.
Incorporation of physiological
and psychological factors of haptic perception in the design of haptic user
interfaces
Physiological Factors: Extensive research has been conducted in
bio-engineering, neurophysiology and neurology domains on the haptic sensory
and perception system. Broadly we can classify the research as (a) research
on sensory apparatus of haptic perception and (b) research on neurological
basis of haptic perception. The human sensory system primarily involves
two types of sensors in haptic sensation: kinesthetic (or manual) and tactile.
Kinesthesia refers to sensations related to gross force/motion of the fingers
or of the hand. Tactile sensations refer to the surface or contact-based
sensations perceived by the hand. Research on tactile sensory system has
shown that
(a) The human hand contains different types of cutaneous sensors and each
of them is specialized to perceive certain sensory stimuli
(b) These sensors are concentrated in different regions of the hand, thereby
specializing different regions of the hand to perceive differential spatial
features. For example the index finger of the hand is specialized to pick
up fine-grain texture while the palm is more suited to pick up shape information.
These findings are important in design and development of haptic systems.
In our proposed system, the physiological specialization of the hand is
encoded in the design. It allows sending differential signals to different
regions of the hand. This system has shown superior performance to a simple
system wherein haptic gloves are sent same signal to each of the tactile
motors.
Psychological Factors: A detailed summary of the psychological
basis of haptics is beyond this document. The key factors that may aid in
development of haptic systems are
(1) Humans are adept at haptic perception in an egocentric reference frame.
In other words humans should be allowed bimanual exploration of objects,
wherein they can manipulate the objects in their hand. This allows humans
to establish a reference point on the object with one hand and move the
other hand over the object to perceive haptic features. Current haptic devices
such as haptic joysticks do not allow egocentric exploration and are hence
limited in their abilities.
(2) Manual Exploratory Procedures are highly structured and humans show
a consistent motor and cognitive strategy of exploration. Lederman and Klatzky,
in their seminal work on haptic exploratory procedures showed that humans
use a consistent movement to perceive a certain feature. For example lateral
movement is used for texture perception. From these movements it is possible
to track the feature a user is trying to perceive and also the saliency
of the feature which is expressed in the order of exploration of features
and number of times it is repeated. This saliency can be encoded into the
system for final haptic rendering.
Design and develop novel methods of haptic rendering
that achieve the goal of virtual/distal environment perception
My approach to design and development of haptic user interfaces combines
three complimentary methodologies for haptic rendering. Traditionally, haptic
or visual rendering is understood as a realistic rendering problem i.e.
make virtual objects look and feel like their real counterparts. This is
by far the most desirable form of virtual reality. However in many cases
it may not be possible to generate the entire gamut of haptic features.
There are two primary reasons for this limitation. Firstly, the haptic devices
available today have limited feedback capabilities as compared to the real
tactile sensory system. The human hand consists of millions of specialized
tactile sensors localized in different regions of the hand, each picking
up sensations in serial due to manual movements. This essentially amounts
to an assembly of parallel specialized sensors perceiving time-varying data.
Present day interfaces have less than 10 tactile feedback motors and clearly
this is not sufficient to provide information to the manual sensor arrays.
The limited tactile feedback is also manifested in the range and type of
feedback that the motors generate. Most of the motors today are capable
of one-dimensional vibratory feedback. While some effort are being made
to develop two-dimensional tactile feedback motors, these efforts and products
are in their infancy. While in the near future the availability of two-dimensional
feedback devices is a distinct possibility, emulating the scale of human
tactile system is anything but trivial. Hence there is a need to develop
innovative approaches that present haptic sensations with these limited
interfaces. Figure 1shows the proposed approach to Haptic Rendering
Figure 1. Proposed Approach to Haptic Rendering
In the proposed approach, haptic rendering is achieved through combination of realistic haptic rendering, non-realistic haptic rendering (haptic visualization) and tactile cueing based or code-based haptic communication. Cueing refers to the methodology of providing tactile cues to individual about global properties of objects. Visualization refers to intuitive modeling of certain haptic features to convey information. Realistic Rendering refers to haptic rendering that mimics real-world haptic sensations. The breakdown of tasks between these three approaches is as follows:
Overall object shape, size, material and texture information through tactile cues
Surface shape and texture patterns: realistic rendering
Other augmentative tactile features: visualization
This division of tasks allows efficient haptic rendering. It allows blind individuals to work in completely haptic environments and perceive objects and their features in 2-5 seconds which is the normal time required for haptic perception. Please click on the links for more information on cueing, realistic rendering and visualization approaches.
ASU Main Website