Gesture recognition system for Kod Tangan Bahasa Melayu (KTBM) using neural network

Abstract

This paper presents simple methods for translating Kod Tangan Bahasa Melayu (KTBM) into voice signal based on subject head and two hand gestures. Different gesture signs made by different subjects are captured using a USB web camera in RGB video stream format with a screen bit depth of 24 bits and a resolution of 320 X 240 pixels. The recorded video of the sign language is divided into number of image frames. Using a simple segmentation technique, the frame image is segmented into three region namely, head region, left hand region and right hand region. After performing the image segmentation, the image frames are converted into binary image format. A simple feature extraction method is then applied and the variations of the features in the subsequent frame are modeled using Discrete Cosine Transform (DCT). The features extracted are associated to the equivalent voice sound and a simple neural network model trained by error prob method is developed. An audio system is used to play the equivalent voice signal from the recognized sign language. Experimental results demonstrate that the recognition rate of the proposed neural network models is about 81.07%.