Non-linear human emotion recognition system using ECG and EMG

Abstract

This thesis focuses on analyzing different nonlinear algorithms to capture the emotional information present in two bio-signals namely Electrocardiogram (ECG) and Electromyogram (EMG). Emotion recognition is one of the emerging research areas in human computer interaction (HCI) and has been widely used in applications such as elearning, smart classrooms, medical applications for patients with autism, Parkinson’s disease etc., Five basic emotional states (happiness, sadness, fear, surprise and disgust) and neutral signal was induced in sixty subjects by means of audio-visual stimuli. The emotion anger was omitted as a result of the pilot study conducted to identify the emotional video clips that could elicit the target emotions in a better way. ECG and EMG signals were pre-processed to eliminate noises that occur due to power line interference and high frequency.QRS complex was then derived from the ECG signals by using a derivative based algorithm. The EMG signals were smoothed and the trend was removed. The emotional frequency was identified by validating the conventional statistical features used for emotion recognition applications using analysis of variance (ANOVA) at different frequency levels. The emotional features extracted from QRS complex and EMG signals at the identified emotional frequency range was classified using four classifiers (Regression Tree, Naïve Bayes, K-nearest neighbor (KNN) and Fuzzy K-Nearest Neighbor (FKNN)). Statistical, Higher Order Statistical (HOS) and non-linear features were obtained from the filtered signals and the signals processed by Hilbert Huang Transform (HHT). A hybrid system replacing Hilbert Transform by Discrete Fourier Transform (DFT) to the signal decomposed and reconstructed by Empirical Mode Decomposition (EMD) was proposed. DFT based method performed better in case of ECG whereas HHT performed better in case of EMG signals. The nonlinear feature Hurst was extracted from the filtered signals using two methods namely Rescaled Range Statistics (RRS) and Finite Variance Analysis (FVA). New methods such as Skewness based RRS, Kurtosis based RRS, Skewness based FVA and Kurtosis based FVA were proposed for computing Hurst by combining HOS with the traditional methods. The features achieved in all the methods were found to be statistically significant (p<0.01). Kurtosis based FVA method performed better for both ECG and EMG signals with an accuracy of 78% and 62% respectively. A system that combines both ECG and EMG signals was obtained by combining the features using principal component analysis (PCA) and multi dimensional scaling (MDS). Principal component analysis (PCA) applied on all the Hurst features derived using FVA for both the signals resulted in an improved accuracy of 82.54%.