Hand gesture recognition for Human Machine Interaction

Researchers of the Grupo de Tratamiento de Imágenes (GTI) of the Escuela Técnica Superior de Ingenieros de Telecomunicación (ETSIT) of the Universidad Politécnica de Madrid (UPM) have developed a hand gesture recognition system based on the use of near-infrared information acquired by the Leap Motion device allowing a more natural and intuitive HMI.

The increase in the availability of electronic devices endowed with interaction has triggered many works in the last few years proposing new ways to achieve the interaction. Many of these solutions look for natural ways of interaction among humans and, thus, are based on voice or gesture interaction. Different alternatives are proposed for this latter case such as gloves, haptic devices, or visual information.

The authors of this project, Tomás Mantecón, Carlos Roberto del Blanco, Fernando Jaureguizar and Narciso García, have proposed a solution based on hand gesture recognition using near-infrared images obtained from a Leap Motion device. This device is specially designed for its use in HMI due to its small size, and, in addition, has been specially designed to operate on a table for the interaction in conventional, in augmented or virtual reality environments. Its output provides near-infrared imagery where closer objects are brighter reducing irrelevant background information for the hand gesture recognition process.

In this work, a system able to distinguish among different gestures, static and dynamic, in just some milliseconds has been developed. As the time between the gesture performance and the system response is very low, a natural and efficient interaction process is achieved. The results have been presented in the paper “A real-time gesture recognition system using near-infrared imagery”, published in the Plos One journal. More info

In addition to the system development, a dataset called “Multi-modal Leap Motion dataset for Hand Gesture Recognition” has been also created. This dataset is composed of different gestures with the objective of testing the system performance. This dataset is intended to be an initial vocabulary of the possible gestures for an HMI process.

This system can be used in a professional environment to interact with robots, or computers, in an augmented or virtual reality environments, or even in home automation environments to allow an interaction with different devices of a connected environment.

GTI in Press

Radio

COPE (minuto 32)

https://www.cope.es/emisoras/comunidad-de-madrid/madrid-provincia/madrid/mediodia-en-cope-mas/audios/ctv-vwp-mediodia-20191008_878097

Academic environment

UPM
https://www.upm.es/?id=e8b9b899b119d610VgnVCM10000009c7648a____&prefmt=articulo&fmt=detail

ETSIT
http://www.etsit.upm.es/otros-elementos/noticias.html?tx_news_pi1%5Bnews%5D=848&tx_news_pi1%5Bcontroller%5D=News&tx_news_pi1%5Baction%5D=detail&cHash=26a437c1d04239ff8a3eec52ba12e725

GTI
http://www.gti.ssr.upm.es/index.php?option=com_content&view=article&id=448

Media 

El Confidencial
https://www.elconfidencial.com/ultima-hora-en-vivo/2019-10-03/desarrollan-un-sistema-capaz-de-encender-un-televisor-con-un-solo-gesto_2607543/

La Vanguardia
https://www.lavanguardia.com/vida/20191003/47790973783/desarrollan-un-sistema-capaz-de-encender-un-televisor-con-un-solo-gesto.html

La Razón
https://innovadores.larazon.es/es/not/desarrollan-un-sistema-capaz-de-encender-un-televisor-con-un-solo-gesto

Efe
https://www.efe.com/efe/espana/efefuturo/un-sistema-capaz-de-encender-una-camara-o-calefaccion-con-solo-gesto/50000905-4079133

Europa Press
https://www.europapress.es/ciencia/laboratorio/noticia-reconocimiento-gestos-tiempo-real-interaccion-robots-20191004134843.html

La Opinión de Málaga
https://www.laopiniondemalaga.es/sociedad/2019/10/06/desarrollan-robots-capaces-reconocer-gestos/1118591.html

La Opinión de Murcia
https://www.laopiniondemurcia.es/sociedad/2019/10/06/desarrollan-robots-capaces-reconocer-gestos/1057842.html

La Opinión de La Coruña
https://www.laopinioncoruna.es/sociedad/2019/10/06/desarrollan-robots-capaces-reconocer-gestos/1441389.html

La Opinión de Zamora
https://www.laopiniondezamora.es/sociedad/2019/10/06/desarrollan-robots-capaces-reconocer-gestos/1195280.html

El Día
https://www.eldia.es/sociedad/2019/10/06/desarrollan-robots-capaces-reconocer-gestos/1014329.html

La Nueva España
https://www.lne.es/sociedad/2019/10/06/desarrollan-robots-capaces-reconocer-gestos/2539667.html

La Región
https://www.laregion.es/content/print/investigadores-espanoles-desarrollan-robot-capaz-reconocer-gestos-milisegundos/20191003213029897555

Diario Información
https://www.diarioinformacion.com/sociedad/2019/10/06/desarrollan-robots-capaces-reconocer-gestos/2193902.html

Diario de Mallorca
https://www.diariodemallorca.es/sociedad/2019/10/06/desarrollan-robots-capaces-reconocer-gestos/1454010.html

DonFelixsSPM
https://donfelixspm.com/2019/10/04/desarrollan-un-sistema-capaz-de-interactuar-con-maquinas-a-traves-de-gestos-en-tiempo-real/

20 Minutos
https://www.20minutos.es/noticia/3788425/0/investigacion-upm-leap-motion-interaccion-maquinas-humanos-gestos-tiempo-real/

Radio One

http://oneradio.com.ar/desarrollan-un-sistema-capaz-de-interactuar-con-maquinas-a-traves-de-gestos-en-tiempo-real/

Tweet