"Advanced face and gesture recognition for visual HMI" 

Tomás Mantecón

E.T.S. Ing. Telecomunicación, Universidad Politécnica de Madrid, November 2018, "Cum Laude".

Ph.D. thesis Directors: Fernando Jaureguizar Núñez y Carlos Roberto del Blanco Adán.

In the last few years, many solutions have been proposed to allow a more natural and intuitive human-machine interaction thanks to the advent of new devices that improve the quality of interaction of keyboards and mouses. Different systems have been designed that make use of different human parts to offer a human-machine interaction as similar as possible to the interaction between humans, using hands or voice. Of special interest are the systems based on hand gestures and visual information, since they are non-intrusive (no sensor is wore by the user) unlike other alternatives as inertial sensors. On the other hand, new authentication systems for these mechanisms of interaction are required in substitution of passwords introduced by keyboard, such as fingerprint recognition, iris identification, or face recognition. The increase of the number of cameras in surveillance environments and embedded in electronic devices (mobiles, tablets, TVs, etc.), has awakened interest in face recognition system based on visual imagery, since no additional sensor is required for the authentication.

This thesis proposes new solutions to solve both face and hand gesture recognition using visual information. With respect to the face recognition systems, three solutions based on the design of feature descriptors adapted to the characteristics of the human face using high-resolution depth-images have been proposed. They allow the face recognition from different perspectives, unlike most of existing works that only accept frontal faces. Depth information makes more difficult the identity theft as a 3D model of the face would be needed for the identification. Two new databases have been created, and made publicly available, to properly evaluate the system, since no high-resolution image databases of faces are available.

With respect to hand gesture recognition, novel solutions are proposed to recognize both static and dynamic hand gestures, which include new descriptors specially designed for depth information that are highly discriminative. These descriptors have been combined with dimensionality reduction techniques to reduce the memory requirements and favor the operation in real time. The proposed systems have been integrated within an Airbus demonstrator as part of the project SAVIER. The demonstrator implements a hand gesture human-machine interaction for a ground control station that commands unmanned aerial vehicles. New databases have been created, and made publicly available, composed by depth and infrared imagery to properly evaluate the system performance. Download here

Tweet