Conduct Virtual Symphony through Natural Interface

Abstract

Virtual reality is an interesting and emerging domain in today’s computer science. It enables users to interact with the virtual world and perform a set of task with the object in the virtual world. Virtual reality applications are commonly used in many areas such as entertainment, engineering, medical, military and defense, education, marketing and business and etc. In most virtual reality applications today, a high degree of realism is an important demand. Realism can be achieved in many different methods such as applying artificial intelligence, use quality graphics, use quality interaction techniques, enabling user to have feedback to all five senses and etc. However achieving high degree of realism is still a difficult task in virtual reality domain. The main goal of this document is to present an approach to improve realism of the virtual reality applications when user interacts with the virtual environment. The proposed approach provides a natural interface to select and grab the attention of virtual humans, which user needs to interact. It has developed a virtual symphony application where user can select musician and provide a set of instructions to control the performance of the selected musician. The proposed system contains a head tracking system, hand tracking and gesture recognition system. Head tracking system is developed using IR sensors and hand gesture recognition process used vision based approach. Virtual musician selection has been done by using user’s head direction. After the selection of the musician user can control selected musician’s performance by using hand gesture commands. According to the user commands virtual musicians should change their performances. Design of proposed system mainly contains four sub systems: Virtual Environment, Scenario controller, Tracking Controller, Sound Controller. Each of these sub systems is responsible for a unique set of tasks. Tracking controller is responsible for performing head tracking, hand tracking and gesture recognition tasks. Scenario controller is responsible for virtual musician selection, managing response of musicians and rendering virtual environment. Sound controller is responsible for select relevant music file, generating 3D sound and managing it. According to the results based on user study, it shows that this proposed approach to communicate with virtual environment is more natural and effective when compared with traditional interaction techniques. Therefore using this kind of an interface improves the realism of virtual reality applications.