Description

Mixed reality (MR) systems refer to the entire broad spectrum that ranges from physical to virtual reality (VR). It includes instances that overlay virtual content on physical information, i.e., Augmented Reality (AR), and those that rely on physical content to increase the realism of virtual environments, i.e., Augmented Virtuality (AV). Such instances tend to be pre-defined for the blend of physical and virtual content. To what extent can MR systems rely on physiological inputs to infer user state and expectations and, in doing, adapt their visualization in response? Measurement sensors for eye and body motion, autonomic arousal (e.g., respiration, electrodermal and heart activity), and cortical activity (e.g., EEG, fNIRS) are widely used in psychological and neuroscience research to infer hidden user states, such as stress, overt/covert attention, working memory load, etc. However, it is unclear if such inferences can serve as useful real-time inputs in controlling the presentation parameters of MR environments. In this thesis project, we will investigate whether this blend can be adaptive to user states, which are inferred from physiological measurements derived from gaze behavior, peripheral physiology (e.g.., electrodermal activity (EDA); electrocardiography (ECG)), and cortical activity (i.e.., electroencephalography (EEG)). In other words, we will investigate the viability and usefulness of MR use scenarios that vary in their blend of virtual and physical content according to user physiology. In particular, we will focus on understanding how physiological readings can passively determine the appropriate amount of visual information to present within an MR system.

You will

• Perform a literature review
• Modify an MR environment
• Adapt existing processing pipeline for EEG and EDA data
• Collect and analyze electroencephalographic (EEG), electrodermal activity (EDA), and electrocardiography (ECG) data
• Summarize your findings in a thesis and present them to an audience
• (Optional) co-writing a research paper

You need

• Strong communication skills in English
• Good knowledge of Unity
• Good knowledge of Python libraries for scientific computing (e.g. Scipy, Neurokit)

References

• Lotte, F., Faller, J., Guger, C., Renard, Y., Pfurtscheller, G., Lecuyer, A., & Leeb, R. (2012). Combining BCI with virtual reality: towards new applications and improved BCI. In Towards practical brain-computer interfaces (pp. 197-220). Springer, Berlin, Heidelberg.
• McGill, M., Boland, D., Murray-Smith, R., & Brewster, S. (2015, April). A dose of reality: Overcoming usability challenges in vr head-mounted displays. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (pp. 2143-2152).
• Fairclough, S. H. (2009). Fundamentals of physiological computing. Interacting with computers, 21(1-2), 133-145.