A framework for Malaysian Sign Language Recognition using deep learning initiatives / Imran Md Jelas

The greatest challenge since the introduction of Malaysian Sign Language (MSL) occur when deaf and hard of hearing people try to communicate using MSL with person without disabilities who do not use MSL. To overcome this communication barrier, a substantial number of studies has been done to produce Malaysian Sign Language Recognition system. Given that MSL is a systematic nonverbal language that utilizes both manual signal and non-manual signal, the employment of a visionbased sign language recognition system denote applicability. A vision-based sign language recognition system utilizes hand direction, wrist orientation and joint angles detection on captured image to capture sign. In 2019, Google introduced MediaPipe a framework bestowed with face detection, hands detection and pose detection suitable for vision-based sign language recognition system. MediaPipe framework can simplified image processing stage tremendously which is crucial in vision-based sign language recognition system. Hence, the main objective of this paper is to develop a Framework for Malaysian Sign Language Recognition using Deep Learning. To achieve this objective, we propose a framework consisting of three main modules namely learning module, training module and detection module. The proposed framework will also be integrated with MediaPipe. Later, Long Short-Term Memory (LSTM) artificial neural network (ANN) is proposed as training algorithm in training module and prediction algorithm in detection module to be used for the development of the actual system based on this proposed framework initiative. LSTM, an ANN can recall both current data and past data. LTSM pertinent for vision-based sign language recognition system especially when continuous image is used as in the proposed framework.