Hand Gesture Vocalization with Home Automation

1Akriti K. Kolhe, 2Aditya Waghmare, 3Vrushabh Moon, 4Ankita D. Mendhekar, 5Prof. V. Surjuse
1, 2, 3, 4Student, 5Assistant Professor, Computer Technology, K.D.K. College of Engineering, Nagpur,
India

Abstract:

Old aged or disabled people who can‘t walk and talk are most sensitive persons and they must be served in a systematic, sophisticated and efficient manner by very little effort. The hand gesture vocalizer with home automation system can be used to serve the old aged or disable persons and to give a full control to them so that they may control all the appliances of home as well as they can talk. Earlier, home automation systems were not cost effective and they were not suitable for aging persons or disable persons. This paper presents an effective solution to overcome these problems. We have designed a low-cost system, which is reliable, efficient and secure gesture operated system for home appliances especially for peoples which are not able to work at home. This system is based on both, software and hardware implementation using ATMEGA microcontrollers. This system is divided in to two main parts namely hand gesture vocalizer and gesture based home automation system. This system uses gesture dependent method.

Index Terms –disabled populations, gesture identification, gesture operated system, gesture recognition, home appliances, and home automation.

I. INTRODUCTION

In the past years, there has been a large increase in the number of speech and hearing impaired and speech disabled persons due to birth defects or accidents. When a speech impaired person speaks to a normal person, then normal person finds it complicated to understand, and asks the disabled person to show gestures for their needs. Disabled persons have their own language to communicate with normal persons; the only thing is that we need to understand their language. Sign language is mostly used by disabled people and it is a communication skill that uses gestures instead of sound to convey meaning simultaneously combining hand shapes, orientations and movement of the hands, arms or body and facial expressions to express easily a speaker’s thoughts. But most of the time normal person finds it complicated to understand this gesture language. The people who can’t speak or have lost the to ability to speak in some accident, it becomes difficult for them to communicate with within the society. To solve this, we have come up with a paper titled ‘HAND GESTURE VOCALIZER WITH HOME AUTOMATION’. It is a social cause paper which greatly reduces the communication gap between the normal and impaired people. Through home automation module, the paper aims to lower the problems faced by aged as well as disabled people by making them capable to control the home appliances easily. They can operate the appliances by providing signs through the gesture glove which internally translates the signs into a particular data. This data can be interpreted by any person.

II. PROBLEM STATEMENT

This paper provides the solution for communication gap between the normal and disabled people. Along with home automation, they are able to implement and control the home appliances as well they can easily communicate with the normal people by the help of this system. The gesture based system provides the following silent features:

• Low cost
• High performance system
• Recognizes gesture
• Provides the output of the gestures in three modules: recognized voice, text on LCD screen and automation control.
• Easy to understand and provide the fluent way for disabled peoples to communicate with the society.

III. LITERATURE REVIEW

Sign language is a language that uses the sign language to convey the meaning. This involves mostly the combination of various shapes, orientation and movement of the hands. Sign language is mainly used by the impaired people. In this paper a wired data gloves is used which is normal cloth fitted with flex sensors along the length of each finger and the thumb. Impaired people can use these data gloves to perform hand gesture and the signals generated will be converted into speech and text so that normal people can understand their expression. [1] The system converts the sign language into voice note, which is easily understandable by blind and normal people. Their sign language is translated into the text format, to facilitate the deaf people as well. This text will be displayed on LCD in order to improve and facilitate more gesture recognition. The GSM modem incorporated with this project help in emergency conditions. It is easy to implement and portable design works on 9V small Radio battery. [2] In the journal, a survey on the different systems implemented for gesture recognition and Gesture Vocalizer. This system facilitates communication between dumb, hearing-impaired and blind people and normal people. It also helps the silent, hearing-impaired and blind to interact among themselves. Some of the applications encapsulated in this paper are gesture recognition and conversion, it is useful to convert sounds into sign language for Mute people, and it can be used for Mobiles for SMS sending. [3] The main advantage of the approach is less computational time and fast response in real time applications. Data transmission to the LCD and recorder makes the user to handle it easily. It can be used very conveniently in public place. [4] The sign language is translated into the text format, to facilitate the impaired people as well. This text is display on LCD. In order to improve and facilitate the impaired people, the more gesture recognition, motion processing unit can be installed and with the help sensor fusion technique, we can accommodate a number of various gestures as well for better and efficient communication. [5] Flex sensor based glove can detect all the movements of hand and the microcontroller based system converts these specified movements into human understandable voice and text display through the LCD display. [6] The Indian sign language translation to text and voice messages using flex sensors are more reliable, user independent and portable system to convert the sign language to text message from which it consumes less power because of the low ultra-power AT89S52 microcontroller design. This text message can be translated to voice using a simple mobile application. It helps to overcome the limited communication between the impaired people with the rest of the world. The signs are converted to letters and it will be displayed on LCD screen and the letters can be transmitted using Bluetooth Module to a smart phone where text to speech conversion takes place. [7] The paper can eliminate the complexity of the previous implementations by eliminating the Bluetooth module and addition of some other modules which are useful for efficient functioning. Some of the major limitations of using complex systems are:

1. Complex structure of the circuit
2. Unnecessary usage of components
3. Increase in the cost the project
4. It requires developing of complex computational algorithms which will detect the gestures
5. Interpreters were very expensive and they are difficult to acquire on a moments’ notice.

These limitations can be overcome by implementing the following

1. Reducing the complexity of the circuit
2. Using simple algorithms for implementations
3. Utilising component usage
4. Using efficient microcontrollers to reduce the processing time and increase the efficiency of the system.

IV. REQUIREMENTS

1. Data Glove:A digital glove is nothing but a regular hand glove which is embedded with 5 flex sensors to the fingers by threading or by using adhesive methods to help the sensors bend properly and to give accurate voltage drop which can be sensed by the microcontroller for appropriate hand gestures.

2. Flex Sensors: The Flex Sensors are fitted on the thumb along with the length of each finger of the digital glove to control and analyze the bending and tilting of the fingers. It is an analog variable resistor sensor which on bending, changes the resistance based upon the measure of a bending in the flex.

3. Arduino UNO: Arduino is an open-source electronics platform based on easy-to-use hardware and software. Arduino boards are used to read inputs the light on a sensor, a finger on a button, and turn it into an output which turns on an LED, publishing something online. The board is equipped with the sets of digital and analog input/output pins that are interfaced to various expansion boards and other circuits.

4. I2C Module: I2C module is a serial protocol for two-wire interface which is used to connect low-speed devices like microcontrollers, I/O interfaces and other similar peripherals in an embedded system. The initial specifications defined on I2C module have a maximum clock frequency of 100 kHz

5. Voice Recorder and Playback: The function of the voice recorder and playback module is to provide speech for the particular gesture information collected from the preceding module of the system. It accepts the data as an input by the prior component of the system and associates the received records with the values that have already been stored in the microcontroller. Now the microcontrollers will fetch the speech output for the respective input gestures, and finally the system gives voice to the specified input gestures.

6. LCD Display: The function of the voice recorder and playback module is to provide speech for the particular gesture information collected from the preceding module of the system. It accepts the data as an input by the prior component of the system and associates the received records with the values that have already been stored in the microcontroller. Now the microcontroller will fetch the speech output for the respective input gestures, and finally the system gives voice to the input gestures.

V. METHODOLOGY

We have data glove in which flex sensors are fitted according to the length of each of the finger. Initially the data glove will fetch input through the bending of the flex sensor. Bending is caused because of the gesture. If the movement is not detected and the values are not obtained by the flex sensors, then we again need to make a perfect gesture. There is a value associated with the bending of each of the flex sensors. Again if that movement is detected and the value is obtained, then the values will be transmitted to the Arduino UNO microcontroller. This data is in the form of analog signals. An I2C module is then connected to the microcontroller so as to convert these analog signals in to digital. All the processing of the data is performed under the modules and the output is obtained into three forms. First output is the recognized voice here the signals are converted in to the voice. The second output is in the text form which is displayed on the LCD screen. And the third output is the automation module where the recorded gestures are used for light and fan control.

VI. EXPERIMENTAL RESULTS

‘Hand gesture vocalizer with home automation’ is a hand gesture based system interface for facilitating communication among normal people and people with speech and hearing disabilities. In this paper, a data glove, which is a normal cloth driving gloves fitted with flex sensors along the length of each of the fingers, is used to provide input. In this paper, an Arduino UNO microcontroller and sensor based gesture to voice and text converter is created so as to recognize some commonly used gestures and convert them into voice message as well as a text message for the benefit of the impaired people. Flex sensor fitted data glove can detect all the movements of a hand and the microcontroller based system converts these specified movements into human recognizable voice, and text is displayed through the LCD screen. Also, the recorded gestures are used to control the automation, which are the light and fan control. To each of the finger gesture, there is a value associated with it which represents the light on/off and fan on/off.

The above figure (fig. 6.2) shows the initial and the success values of the gesture vocalizer system. The success values are the actual resultant values of the whole hand gesture vocalizer module which outputs the gesture detection and home automation control. The table below (table 1) represents all the result values, that is, success value with the success rate of the whole project paper.

Test No.	Testing finger gesture	Success value	Rate of success (%)	Output modules
LCD screen	Playback module	Automation
1	Index	970	95	Yes	Yes	No
2	Mid	920	90	Yes	Yes	No
3	Ring	950	92	Yes	Yes	No
4	Little	940	96	Yes	Yes	No
5	Index	970	92	No	No	Yes
6	Mid	920	94	No	No	Yes
7	Ring	950	97	No	No	Yes
8	Little	940	98	No	No	Yes

Table 1 Test results of gesture recognition

VII. CONCLUSION

The proposed method translates sign language to speech and text automatically and satisfy impaired by conveying the thoughts. The system overcomes the real time difficulties of impaired people and improves their communication. System efficiency is improved with the help of Arduino UNO microcontroller and I2C module which help in the communication. By implementing this system, speaking dream of impaired people becomes real. Compared with existing system it is possible to carry this system to any places. The main advantages of this approach are efficiency and less computational time, and fast response in real time applications has been achieved. Due to the data transmission to the LCD and recorder, it makes the user to handle it easily. It can be used in public places very conveniently and effortlessly. In future ,the system can be made more compact by mounting all the modules on same data glove itself .The presented system may not give 100% accurate result due to high sensitivity of sensors. Hence, more development is needed for flex sensors. System is trained on a limited database and should be made capable to handle more gesture.

VIII. FUTURE SCOPE

1. The data glove can more over be used to control the appliances like computer, TV, etc.
2. It can be used for sending messages from mobile phones
3. More languages can be fed so that local people can also understand.
4. Signals from animals can also be gathered.
5. A system can be designed to make the whole project wireless.
6. Sensor based gesture vocalizer to give a better desired output
7. A more compact device fetch that all the modules are mounted on the same data glove.

REFERENCES

1. International Journal for Research in Applied Science & Engineering Technology (IJRASET), ISSN: 2321-9653; IC Value: 45.98; SJ Impact Factor: 7.177, Volume 7 Issue 9, September-2019 Survey on Gesture Vocalizer for Speech and Hearing Impaired, 1Vrushabh Moon, 2Aditya Waghmare, 3Akriti K. Kolhe, 4Ankita D. Mendhekar, 5 Prof. V. Surjuse, 1, 2, 3, 4Student, 5Assistant Professor, Computer Technology, K.D.K. College of Engineering, Nagpur, India.
2. Karibasappa R and Choodarathnakara A L Government Engineering College, Kushalnagar, Karnataka, INDIA Ishana M E, Thanuja C, Basavaraju M K and Sunitha C Government Engineering College, Kushalnagar, Karntaka, INDIA, Smart Glove Based Gesture Vocalizer for Deaf and Dumb, International Journal of Management, Technology And EngineeringVolume 8, Issue XI, NOVEMBER/2018 ISSN NO : 2249-7455.
3. International Journal for Research in Applied Science & Engineering Technology (IJRASET), A Review on Gesture Vocalizer Deena Nath1, Jitendra Kurmi2, DevekiNandan Shukla3 1, 2, 3Department of Computer Science, Babasaheb Bhimrao Ambedkar University Lucknow, ISSN: 2321-9653; IC Value: 45.98; SJ Impact Factor: 6.887 Volume 6 Issue IV, April 2018, IJRASET (UGC Approved Journal:All Rights are Reserved 2990.
4. Dalal Abdulla, Shahrazad Abdulla, RameesaManaf and Anwar H. Jarndal, 2017,” Design and Implementation of a Sign-to Speech and/or Text System for Deaf and Dumb People”, Institute of Electrical and Electronics Engineers (IEEE).
5. Third IEEE International Conference on Computational Intelligence and Communication Technology, IEEE-CICT, Suraksha Devi and Suman Deb, 2017, “Low Cost Tangible Glove for Translating Sign Gestures to Speechand Text in Hindi Language”.
6. IEEE Ecuador Technical Chapters Meeting (ETCM), 1Santiago Aguiar, 2Andres Erazo and 3Sebastian Romero, 2016, “Development of a Smart Glove as a Communication Tool for People with Hearing Impairment and Speech Disorders”.
7. International Journal of Current Trends in Engineering & Research (IJCTER) e-ISSN 2455–1392 Volume 2 Issue 5, May 2016 pp. 327 – 336, IJCTER-2016, Scientific Journal Impact Factor:3.468, All rights Reserved 327 Gesture Vocalizer for Dumb People.
8. International Journal of Innovative Research in Computer and Communication Engineering, An ISO 3297: 2007 Certified Organization, K. V. Fale1 , Akshay Phalke2 , Pratik Chaudhari3, Pradeep Jadhav4, Smart Glove: Gesture Vocalizer for Deaf and Dumb People, Volume 4, Issue 4, April 2016 Copyright to IJIRCCE DOI: 10.15680/IJIRCCE.2016.
9. International Journal of Current Trends in Engineering & Research (IJCTER) e-ISSN 2455–1392 Volume 2 Issue 4, April 2016 pp. 321 – 336 Scientific Journal Impact Factor : 3.468 FACIAL POINT DETECTION AND EMOTION RECOGNITION FOR A HUMANOID ROBOT A.V.Kiranmai1, S.Karimulla2 Asst. Professor, Dept. of E.C.E, SVEW, Tirupathi.
10. Kiran R, Department of Electronics and Communication Engineering, College of Engineering Trivandrum, India “Digital Vocalizer System for Speech and Hearing Impaired” International Journal of Advanced Research in Computer and Communication Engineering, Vol. 4, Issue 5, May 2015.
11. International Journal of Advanced Research in Computer and Communication Engineering, Vol. 4, Issue 5, May 2015, Sachin Bhat, Amruthesh M, Ashik, Chidanand Das, Sujith, ”Translating Indian Sign Language to text and voice messages using flex sensors”.
12. National Conferenceon Synergetic Trends in engineering and Technology: Impact of Industry & Society (STET-2014) InternationalJournal of Engineering and Technical Research (IJETR) ISSN: 2321-0869, Special Issue, Naveen Kumar Toshawar, Chandni Jain, “Intelligent Control System using Gesture Sensing”.
13. Vajjarapulavanya, Akulapravin, M.S., Madhanmohan, “Hand Gesture Recognition and Voice Conversion SystemUsing Sign Language Transcription System” IJECT Vol. 5, Issue 4, Oct – Dec 2014.
14. Journal of Engineering Research and Studies, Vol. II, Issue IV, October-December, 2011/113-115, Bhavina Patel, Vandana Shah, Ravindrakshirsagar, “Microcontroller Based Gesture Recognition System for the Handicap People”.