In-situ recognition of hand gesture via Enhanced Xception based single-stage deep convolutional neural network

•The Hybrid-SSR framework is proposed to perform real-time hand gesture recognition.•The Enhanced Xception (E-Xception) architecture is utilized as a backbone network.•Mitigates the misclassification problem encountered by the conventional models.•The proposed model is evaluated on MITI-HD, NUSHP-II...

Full description

Saved in:

Bibliographic Details
Published in:	Expert systems with applications 2022-05, Vol.193, p.116427, Article 116427
Main Authors:	Bose, S. Rubin, Kumar, V. Sathiesh
Format:	Article
Language:	eng
Subjects:	Artificial neural networks Classification Convolutional neural network Datasets Deep learning Feature extraction Gesture recognition Graphical user interface Hand gesture recognition Hybrid systems Machine vision Mathematical models Neural networks Object detection Optimization Optimization techniques Parameters Telesurgery
Online Access:	Get full text
Tags:	Add Tag No Tags, Be the first to tag this record!

Description
Summary:	•The Hybrid-SSR framework is proposed to perform real-time hand gesture recognition.•The Enhanced Xception (E-Xception) architecture is utilized as a backbone network.•Mitigates the misclassification problem encountered by the conventional models.•The proposed model is evaluated on MITI-HD, NUSHP-II and Senz-3D datasets. In-situ detection and recognition of handgestures in an unconstrained environment is a crucial task in the field of machine vision and video recognition. Hand gesture recognizing task is perhaps more complicated, with varying lighting conditions and background. Robust, accurate hand detection and classification becomes necessary to facilitate the Graphical User Interface, Tele-surgery, etc. In this paper, a Convolutional Neural Network (CNN) based Hybrid Single Stage Recognition (Hybrid-SSR) framework is proposed for real-time hand gesture recognition. The time-efficient Enhanced Xception (E-Xception) CNN model is proposed for feature extraction. The framework is evaluated using three separate datasets (NUSHP-II, Senz-3D, and MITI-HD). The performance of the framework is evaluated for the range of IoU from 0.5 to 0.95. In comparison with the conventional single-stage hand gesture recognition system, the Hybrid-SSR model resulted in higher precision values (99.60 % on AP0.5, 97.80 % on AP0.75, and 88.20 % on AP0.5:0.95) for the MITI-HD dataset. The Hybrid-SSR model with Adam Optimizer surpasses the other optimization techniques. The proposed model achieved comparatively higher precision and recall values for focusing parameter (γ) = 2 and weighting parameter (α) = 0.5. The misclassification problem encountered by the conventional single-stage models is mitigated by implementing the focal loss function in the classification sub-network. The computation cost of Hybrid-SSR (12 ms) is significantly lower compared to other single-stage hand gesture recognition frameworks.
ISSN:	0957-4174 1873-6793