Indian Sign Language recognition system using SURF with SVM and CNN

Hand signs are an effective form of human-to-human communication that has a number of possible applications. Being a natural means of interaction, they are commonly used for communication purposes by speech impaired people worldwide. In fact, about one percent of the Indian population belongs to thi...

Full description

Saved in:

Bibliographic Details
Published in:	Array (New York) 2022-07, Vol.14, p.100141, Article 100141
Main Authors:	Katoch, Shagun, Singh, Varsha, Tiwary, Uma Shanker
Format:	Article
Language:	eng
Subjects:	Bag of visual words (BOVW) CNN Google speech API Hand sign recognition Indian sign language (ISL) Pyttsx3 SURF features SVM
Online Access:	Get full text
Tags:	Add Tag No Tags, Be the first to tag this record!

Description
Summary:	Hand signs are an effective form of human-to-human communication that has a number of possible applications. Being a natural means of interaction, they are commonly used for communication purposes by speech impaired people worldwide. In fact, about one percent of the Indian population belongs to this category. This is the key reason why it would have a huge beneficial effect on these individuals to incorporate a framework that would understand Indian Sign Language. In this paper, we present a technique that uses the Bag of Visual Words model (BOVW) to recognize Indian sign language alphabets (A-Z) and digits (0–9) in a live video stream and output the predicted labels in the form of text as well as speech. Segmentation is done based on skin colour as well as background subtraction. SURF (Speeded Up Robust Features) features have been extracted from the images and histograms are generated to map the signs with corresponding labels. The Support Vector Machine (SVM) and Convolutional Neural Networks (CNN) are used for classification. An interactive Graphical User Interface (GUI) is also developed for easy access.
ISSN:	2590-0056 2590-0056