Interactive dashboard with visual sensing and fast reactivity

Abstract

These days, technology is growing rapidly, and the market has been introduced with lots of fascinating ways to interact with computers. The advancement of deep learning models and hardware technology also enables more applications with fancy features to be built. The importance of hand gesture recognition has increased due to the prevalence of touchless applications. However, developing an efficient recognition system needs to overcome the challenges of hand segmentation, local hand shape representation, global body configuration representation, and a gesture sequence model. This paper proposed an interactive dashboard that could react to hand gestures. This is also an initiative of the Tunku Abdul Rahman University College (TAR UC) Smart Campus project. Deep learning models were investigated in this research and the optimal model was selected for the dashboard. In addition, 20BN Jester Dataset was used for the dashboard development. To set up a more user-friendly dashboard, the data communication stream between the captured input stream and commands among the devices were studied. As to achieve higher responsiveness from the dashboard, evaluation on data communication protocols which were used to pass the input data were included in the study.