An intelligent gesture recognition system

Abstract

Information and knowledge are expanding in quantity and accessibility. However, people with functional limitations, such as hearing impaired, often left out of conversation where there are wide communication gaps between them with the ordinary people. The sign language is the fundamental communication method between people who suffer from hearing defects. In order for an ordinary people to communicate with hearing impaired community, a translator is usually needed to translate the sign language into natural language. This project presents a simple method for converting sign language into voice signal using features obtained from the hand gestures. Using a camera, the system receives sign language video from the hearing impaired subject in the form of video streams in RGB (red-green-blue) colour with a screen bit depth of 24-bits and a resolution of 320 x 240 pixels. For each frame of images, two hand regions are segmented and then converted into binary image. Feature extraction model is then applied on each of segmented image to get the most important feature from the image. Artificial Neural Network (ANN) provides alternative form of computing that attempts to mimic the functionality of the brain. A simple neural network model is developed for sign recognition directly from the video stream. An audio system is installed to play the particular word for the communication between the ordinary people and hearing impaired community.