The first question that aroused my curiosity was, ‘What exactly is a ‘gesture?’ According to a dictionary, gesture is a sequence of postures that are connected by motions over a short period of time. I did not find that really helpful. I’ll explain it in an easier way—a gesture is a form of non-verbal or non-vocal communication. So your messages are communicated by visible bodily actions like waving, swiping, facial movements or even pointing to the sky.
What is gesture recognition
This technology enables devices to interpret human gestures using sensors, whose data is then processed using mathematical algorithms. The system is built to identify specific human gestures, and then use them either to convey information or to control a device. The gestures used have to be intuitive, simple and universally acceptable to ensure they are easily adopted by users.
Gesture recognition makes computers more accessible for the physicall
y-challenged and also makes interaction more natural in the 3D virtual world or in gaming. In today’s world, gesture recognition is an evolving technology that can be seen all around us. The latest phones, computers, gaming consoles, and even TVs now feature it.
How it works
The human body’s movements are read by an imaging sensor like a camera, which then sends that data to a computer. The computer uses the gestures captured as inputs, processes this input to understand what was gestured, and then sends the answer as a command to the currently running application or control device.
Let us look at an example of a person clapping his hands in front of a camera. When this clapping gesture data is processed by the connected computer, it could produce the sound of cymbals, as programmed. The specialty of this technology is that users do not have to wear any special equipment or attach any device to their bodies, as is the case with some alternative technologies.
Sensing the gestures
When you gesture at your computer or mobile device, it obviously needs a way to ‘see’ or ‘sense’ what you are doing. This is where sensors come in. There are a number of sensing technologies that can be considered, but we’ll stick to the popular ones here.
Accelerometers. Also known as inertial or acceleration sensors, these are used to detect the changes in force resulting from the fall, motion, tilt, shock, positioning or vibration of the body they are connected to, with respect to time. These measure linear acceleration, so if the device is in a free fall, the accelerometer will give a ‘0’ value for acceleration.
Application. In the case of a smartphone, the direction of the screen changes as soon as we tilt or turn the phone. The direction in which we hold the phone (horizontal or vertical) is sensed by an accelerometer and the information in the form of electrical signals is sent to the screen, which automatically adjusts itself either in landscape or in portrait mode. So, here, contact gesture recognition plays an important role.
Accelerometers have paved the way for many games to be developed for smartphones, especially for some of those sports simulation games that let you use the phone as a steering wheel or a fishing pole. To get an idea of the range of accelerometers that you can select from, refer to Table I.
Gyroscopes. Gyroscopes, also popularly known as gyros, have had a major impact in the world of gesture recognition. Their ability to measure the orientation of a device enables them to measure both vertical and horizontal rotation. This has brought smoothness and preciseness to the operation of a mobile device. The next time you hold a phone on its side, and the screen transforms into landscape mode automatically, you know which component to thank.
Application. In the case of a smartphone or a tablet, adding a gyro and an accelerometer allows the device to sense motion on six axes including roll, pitch and yaw—which helps you fly your airplane in a flight simulator game on a phone. So if accelerometers have given more business to game developers, gyros have helped them be more innovative.
Bend sensors. These are also known as flex sensors. As we know, even finger movements are considered as gestures, so bend sensors help us to identify those gestures by recognising finger postures. They work on the principle that resistance increases with an increase in the bend.
Data gloves. Gesture recognition technology has not only been implemented in smartphones and tablets, but also in television sets and computers. In order to implement hand gesture recognition on these devices, data gloves have been developed. These gloves identify the finger and hand gestures in real time scenario. These are wired either to a TV set or to a computer and then worn by the person operating the device. These act as an input device and eliminate the use of a mouse or a keyboard.
Fantastic work by sneha ambastha ,this article was definitely very much helpful thank you EFY
Thank you Rishabh for your valuable comment.