Word-Level ASL Recognition and Trigger Sign Detection with RF Sensors

Current research in the recognition of American Sign Language (ASL) has focused on perception using video or wearable gloves. However, deaf ASL users have expressed concern about the invasion of privacy with video, as well as the interference with daily activity and restrictions on movement presente...

Full description

Saved in:
Bibliographic Details
Main Authors: Rahman, M. Mahbubur, Kurtoglu, Emre, Mdrafi, Robiulhossain, Gurbuz, Ali C., Malaia, Evie, Crawford, Chris, Griffin, Darrin, Gurbuz, Sevgi Z.
Format: Conference Proceeding
Language:English
Subjects:
ASL
Conferences
deep learning
Gesture recognition
Human computer interaction
Interference
micro-Doppler
Privacy
Radio frequency
RF sensing
sign language
Training
Online Access:Request full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Current research in the recognition of American Sign Language (ASL) has focused on perception using video or wearable gloves. However, deaf ASL users have expressed concern about the invasion of privacy with video, as well as the interference with daily activity and restrictions on movement presented by wearable gloves. In contrast, RF sensors can mitigate these issues as it is a non-contact ambient sensor that is effective in the dark and can penetrate clothes, while only recording speed and distance. Thus, this paper investigates RF sensing as an alternative sensing modality for ASL recognition to facilitate interactive devices and smart environments for the deaf and hard-of-hearing. In particular, the recognition of up to 20 ASL signs, sequential classification of signing mixed with daily activity, and detection of a trigger sign to initiate human-computer interaction (HCI) via RF sensors is presented. Results yield %91.3 ASL word-level classification accuracy, %92.3 sequential recognition accuracy, 0.93 trigger recognition rate.
ISSN:2379-190X
DOI:10.1109/ICASSP39728.2021.9414063