User Stress Detection through Emotion Understanding

Abstract

“Deadlines” is a word that almost all employees dread of hearing. In the modern day where deadlines are in abundance, employees are expected to work harder than ever before. Stress is body’s way of reacting to emergency situations and in this excited state human body cannot endure for a long period of time, therefore stress should be avoided. The focused subjects of this research are employees of an IT company where they work for long hours in front of the computer, having tight deadlines to work with. This research primarily focuses on getting physiological data in an non-invasive manner. A customised application written in Java controls the webcam and captures video feed of users then extracts still images from it. Still images are parsed in order to find the facial feature points of the user using Luxand API. Next, the extracted feature points are converted to facial features using a program written in Python, which is used to find the emotion of the participant in the still image. Since all images extracted from the videos are unlabelled, an unsupervised learning technique is most appropriate. A simple yet effective unsupervised learning algorithm is K-means algorithm. K-means algorithm is used along with CK+ database which has over 10,000 of still images of people portraying 7 different emotions. CK+ database consist of still images covering emotions of anger, contempt, disgust, happy, sad, surprise and fear. CK+ database is the training dataset for the K-means model. To identify the clusters, some labeled data of CK+ database images are parsed through the trained K-means model. The trained K-means model resulted in 8 clusters corresponding to the the afore mentioned 7 emotions (in CK+ database) along with the neutral state of emotion. It should be noted that, different types of pre-processing steps and parameter tuning was done to achieve optimal results. Although, the resulting clustering was unable to represent each k-mean cluster with a unique emotion, results show that similar negative emotional states such as angry, contempt and disgust cluster together, thus, showing the potential of this research.This research can be further enhanced with the use of wearable devices such as smart watches in a non-invasive manner to include other physiological measures such a heartbeat, to capture stress levels and emotions in additional to facial features.