Doctoral Consortium

The IMX 2021 Doctoral Consortium (‘the DC’) is a one-day, single-track event that will provide an opportunity for 8 PhD students to present, explore and develop their research interests, under the mentorship of a panel of 13 distinguished researchers, including 2 one-on-one sessions. Distinguished researchers at the DC are a mix of gender: (60% male, 40% female), and represent both academia and industry. Both students and the DC distinguished researchers lived/grew up across different parts of the world including Africa, Asia, Europe, and the Americas.

Overall IMX DC is aimed at:

• Providing a setting for students to discuss their research with distinguished researchers and doctoral students outside their own institution;

• Offering mentorship, individual feedback and fresh perspectives about each students’ current research and suggest possibilities for future research directions;

• Promoting the development of a supportive community of scholars and a spirit of collaborative research; and

• Contributing to the conference goals through interaction with other researchers and conference events.

Each accepted submission will be presented at the DC before the main conference, and also as a poster at the conference, enabling wider visibility of their work. All DC students and distinguished mentors  have received complimentary conference registration that includes free access to the conference program during the conference. 

Understanding Immersion Experience When Using Virtual Reality for Foreign Language Learning

Hye-Kyung Bae, Dept. of Information Systems, University of Maryland Baltimore County, Maryland, United States

Abstract: One of the most effective ways to learn a foreign language is by immersion in its native cultures. However, in practice, few students have access to these kinds of experiences. Virtual Reality (VR) technology has the potential to expand access allowing more students to have similar immersion experiences. These experiences are more engaging than conventional methods such as passively watching videos or interacting with two-dimensional user interfaces. However, are they more conducive to learning? My dissertation project is to examine students’ immersion experience in different VR conditions for their foreign language learning through three studies: guided naturalistic VR condition, unguided and guided naturalistic VR conditions, and naturalistic and artificial VR.

 A Quality of Experience Evaluation of Instruction Formats for Procedure Training in Augmented Reality

Eoghan Hynes, Dept. of Computer & Software Engineering, Athlone Institute of Technology, Ireland 

Ronan Flynn, Dept. of Computer & Software Engineering, Athlone Institute of Technology, Ireland 

Brian Lee, Software Research Institute, Athlone Institute of Technology, Ireland 

Niall Murray, Dept. of Computer & Software Engineering, Athlone Institute of Technology, Ireland

Abstract: Augmented reality (AR) is widely regarded as a promising training platform. AR integrates real and virtual information into a single scene. This reduces attention changes between workpiece and instruction. Error correction in the form of instruction verification feedback using computer vision can ensure correct learning during training. AR based training can be personalized to suit trainee requirements from novice to expert. Multimedia instruction formats can reduce extraneous cognitive load on the trainee compared to single medium instruction formats such as text only. This leaves the trainee with more cognitive resources for understanding and learning. However, multimedia instruction formats tend to consume more resources than static text only instruction formats. This is a concern for mobile AR devices where processing and power resources are limited. This work will evaluate the influence of instruction format resource efficiency on learning in an AR procedure training application. This will be achieved in a between groups study design where animated 3D instructions are compared against static 2D text-based instructions. Learning will be objectively evaluated in a post training recall phase. The extrinsic cognitive load resulting from the different instruction formats is the independent variable that will influence learning as measured from each individual participant’s baseline. The participants cognitive load will be measured implicitly by means of eye-tracking using the in-built Hololens 2 mixed reality headset eye tracking sensors, and by micro facial expressions (MFEs). An application profiler will record digital resource consumption influenced by the graphical and computation requirements of the different instruction formats. An AR Rubik’s Cube solving application will be used for this evaluation. This work will focus on the influence of cognitive load on AR user quality of experience (QoE). The user’s QoE will be measured implicitly via emotion by means of the participant’s micro-facial expressions and physiological signals. The participant’s QoE and cognitive load will be subjectively reported in post experience questionnaires. This work will also evaluate the utility of AR instruction verification feedback, self-paced training, and the influence of these on user QoE.

Real-time Anxiety Prediction in Virtual Reality Exposure Therapy

Deniz Mevlevioğlu, University College Cork, Ireland

Sabin Tabirca, University College Cork, Ireland

David Murphy, University College Cork, Ireland

Abstract: Detection of anxiety patterns in real-time within a virtual reality environment has many uses for medicinal, psychological or entertainment purposes. Virtual reality exposure therapy (VRET) is a therapy method that is quickly rising in popularity, and a built-in way to monitor anxiety levels within VRET applications can contribute to the therapy by providing physiological feedback from the user. This feedback can be used to make meaningful adjustments to context such as increasing exposure levels as user anxiety decreases. For the measurement of physiological signals within Virtual Reality applications, on-body biosensors are generally preferred due to mobility concerns. These biosensors can, however, be susceptible to noise due to movement and it is hard to extract information from a single type of signal. As a countermeasure, this study uses multimodal data and machine learning. The goal of the study is to integrate these signals into a virtual reality experience and accurately assess anxiety levels in real-time by examining patterns across different types of measurements and using a neural network to process information and reduce the effect of noise.

Redesigning Digital Peer-to-Peer Payments for Social Connections: A Sociotechnical Approach

Lingyuan Li School of Computing, Clemson University, Clemson, South Carolina, United States

Abstract: My proposed dissertation research investigates the social and interactive aspects of digital peer-to-peer (P2P) payments and seeks to redesign such technology for supporting both secure transactions and nuanced social interactions/connections through a sociotechnical approach. Specifically, my goal is to i) reveal and elaborate the multidimensional influences of digital P2P payments on both financial experiences/processes and everyday social interactions, ii) design and develop prototypes that highlight the importance of the interplay of financial and social engagement when designing digital P2P payments platforms. My research will not only contribute to better understanding new and more complicated social phenomena and dynamics emerging in today’s digital economy but also benefit the HCI community by informing future research on computer-mediated communication through financial technologies.

Virtual Reality And Collaborative Learning In Higher Education A Research Thesis Proposal

Nesse van der Meer N Delft Technical University, Centre for Education and Learning, Delft, Zuid Holland, Netherlands

Abstract: For the past forty years, research has revealed many ways in which Computer-Supported Collaborative Learning (CSCL) can be utilized to support and enhance education. Similarly, Virtual Reality (VR) technology has proven promising for different educational purposes. These affordances, however, tend to change when the technology behind them advances; as such, research should continue to examine the effects how these advances affect the pedagogy. With VR technology continuously expanding, its potential for Collaborative Learning (CL) requires further studying. This research project aims to better understand how different aspects of VR can be used to support key dimensions of successful CL. In doing so, the research project hopes to provide guidelines as to how these VR aspects can be implemented to facilitate and enhance collaboration between group members in an educational setting.

Non-Visual Web Design: Enhancing User Experience (UX) Through Aesthetic Semantics

Labake Odushegun University of Dundee, Dundee, Scotland, United Kingdom

Abstract: This project will improve the ability of people to experience non-visual web content by developing an understanding of the relationship between web based visual aesthetic semantics and their auditory counterparts. Current non-visual methods of presenting web applications focus on text-based information transfer and do not take into consideration the aesthetic design or intended emotive journey that users should feel. As a result, users experience high cognitive load, information overload and a reduced ability to fully engage with aesthetic-rich websites in non-visual interfaces. A potentially innovative way to retain the informative data of visual web components for use in non-visual systems, is to correlate the semantic meaning of visual aesthetics to non-visual aesthetics of semantic equivalence. This research has three specific aims:1. Discover which visual and non-visual aesthetics have the strongest impact on affective semantics; 2. Establish if analogous affects exist between visual and non-visual web aesthetics; 3. Develop a framework which delivers analogous informative value in both visual and non-visual web pages using affective aesthetic semantics.The objectives of this research are as follows: 1. Carry out an online quantitative survey to determine affective responses to atomic web aesthetics; 2. Develop a JS library of atomic affect ratings, which quantifies singular and compounded aesthetic web elements, and can be used to correlate said atomic ratings to audio with analogous ratings; 3. Produce a scalable and extendable JavaScript framework which utilises the core concept of aesthetic semantics and atomic affect rating to enhance UX design in non-visual interfaces.

Toward an Understanding of Flow Cost in Entertainment Media Use

Giang V Pham University of Illinois at Urbana-Champaign, Institute of Communications Research, Urbana, Illinois, United States

Abstract: Flow is a highly desirable experience in entertainment media use because it can facilitate escapism and enjoyment. This paper proposes a research program that challenges the view that the outcomes of flow are not always positive, rather, they are dependent on the context in which flow occurs. When occurring under the pursuit of multiple goals in a resource-limited context, the full absorption of cognitive and attentional resources due to flow can potentially divert the user’s resources (attention, cognitive effort, time) away from goals outside of the media activity, imposing resource costs to the performance of such goals. Employing lab experiments, this research will examine people’s attention, task performance, and affect when engaging in the flow-inducing media use and subsequent activities. Findings from this research will inform the design of intelligent systems such as management tools or virtual assistants that can reduce the cost of negative flow for users.

Understanding the Voluntary Moderation Practices in Live Streaming Communities

Jie Cai New Jersey Institute of Technology, Newark, New Jersey, United States

Abstract: The moderation task in the live streaming community is challenging due to the interactivity and ephemerality of live text-based communication in the chat. Moderators had to make decisions with time constraints and less instruction, experiencing information overload and emotional toll. I aim to understand their decision-making process, identify the challenges during the process, explore the relationship with other stakeholders in the community. I applied mixed methods (interview, survey, observation) to explore these issues. My dissertation focused on content moderation in interactive media, and the results can potentially provide guidance to content moderation in interactive media platforms with high interactivity and synchronicity.


Airi LampinenAssociate Professor, Stockholm University, Sweden
Casey FieslerAssistant Professor, University of Colorado Boulder, US
David GeertsResearch Manager, KU Leuven, Belgium
Min Hane AungLecturer, University of East Anglia, UK
Hao chuan WangAssociate Professor, University of California Davis, US
Jofish KayeSenior Director of Interaction Design & AI, Anthem, US
Jonathan HookSenior Lecturer, University of York, UK
Louise BarkhuusAssociate Professor, IT University of Copenhagen, Denmark
Niels HenzeProfessor, University of Regensburg, Germany
Pablo CesarProfessor, TU Delft, Netherlands
Stefan SchneegassJunior Professor, University of Duisburg-Essen, Germany
Vinoba VinayagamoorthyResearch Engineer, BBC R&D, UK
Yvette WohnAssociate Professor, New Jersey Institute of Technology, US
Benjamin WeyersJunior Professor, University of Trier, Germany