A phoneme based sign language recognition system using interleaving feature and neural network

Abstract

A sign language is a language which, instead of acoustically conveyed sound patterns, uses visually transmitted sign patterns. Sign languages are commonly developed in hearing impaired communities, which can include interpreters, friends and families of deaf people as well as people who are deaf or hard of hearing themselves. Developing a sign language recognition system will help the hearing impaired to communicate more fluently with the normal people. This paper presents a simple sign language recognition system that has been developed using skin color segmentation and Neural Network. A simple segmentation process is carried out to separate the right and left hand regions from the image frame and in the preprocessing stage the vertical interleaving method is used to reduce the size of the image. The 2D moment of the right and left hand interleaved image is obtained as features. Using the interleaved 2D-moment features, a simple neural network model was developed. The system has been implemented and tested for its validity. Experimental results show that the system has a recognition rate of 91.12%.