Sign language recognition using You Only Look Once-Neural Architecture Search / Muhammad Imran Nasharudin

The American Sign Language (ASL) is a nonverbal communication language that uses visual sign patterns formed with the hands or any part of the body and is usually utilized by persons who have hearing or listening disabilities. The deaf and mute people have difficulty communicating their thoughts, needs, and feelings through spoken language. There is a need to have an alternative method based on computer technology for those who are deafen or hard of hearing people since vocal communication is not available to them. One of the issues of computer-based sign language recognition is latency which creates delay in executing the interpretation of certain gestures. To balance latency vs. throughput, the architecture is discovered automatically using a Neural Architecture Search (NAS) technology called AutoNAC. The innovative features of YOLO-NAS include the quantization aware modules QSP and QCI, which combine re-parameterization for 8-bit quantization to minimize accuracy loss during post-training quantization. The architecture is designed to identify tiny objects, increase localization accuracy, and improve the performance-per-compute ratio, making it appropriate for real-time edge-device applications. As using YOLO-NAS for the sign language recognition, we succeeded to deploy average of detection percentage of 86% of all sign language alphabets. YOLO-NAS networks are successfully used in sign language recognition, with a reported 96.41 (mAP@50).