CognitiveEmotional User Correction for Multimedia Interactions Using Visual Attention and Psychophysiological Signals

Abstract

Today, the multimedia interaction domain is a developed area not only focusing on the presentation of information to the human user, but also get the human involved in computer modeled realworld scenarios, such as computer games and elearning. In this relation, the computer can only process symbolic information and the knowledge is representation oriented. However, humans are biological creations having basically three different aspects: brain activation, cognitive behavior and emotional reaction. Currently, under human computer interactions, only the cognitive behavior of the human is recognized by the computer. As a result, humans do not show the same involvement when they are attached to real world activities and when the activities are decided by the computer. In multimedia computing, this problem is termed as perceptionsemantic gap, arguing the main reason for this problem is the semantic incompatibility between the knowledge representation of the computer and the internal representation of the human user. However, modern cognitive science explains that human behavior can not be described without considering the emotional reaction revealing many relationships between cognition and emotion. Therefore, researchers are trying to recognize emotions using facial expressions of the human user. However, this approach does not provide a correct relationship, because the facial muscles are under conscious control of the subject, so the expressions can easily be faked. Psychophysiological explanations of emotional reaction describes better relationships between a person's true psychological behavior and his/her bodily (physiological) changes. Therefore, the work of this research tries to capture these bodily signals to determine the emotional involvement of the subject. Humans exist in a world with unlimited knowledge dynamics. Therefore, selective attention helps humans to filter out unwanted information from their senses. The visual attention plays an important role in this effort, because it gives a directional focus to the attended information in the outside world. Therefore, the research tries to capture the visual attention using biopotentials generated as a result of eye movement activity of the subject, which is a lowcost solution compared to sophisticated and expensive techniques used today that returns the actual spacial coordinates of the eye focus.The attention and most other cognitive aspects of the human are controlled by a mental phenomenon called the consciousness. Yet, the consciousness is beyond any scientific explanation and can only be linked with cognitive emotional aspects using philosophical models. Therefore, the study also includes a proposal of a coherent model of human psyche which is then used to develop a psychophysiological evaluation framework that can be used to determine discrete levels of involvement of a subject to a multimedia interaction. Results of the experiments show the validity of the framework.