Technical Program

This paper explores how novel possibilities for personalisation and interactivity in video media, posed by developments in responsive video technology, can be applied to support patients as they make complex decisions about their health and care. We present the stakeholder-centred design of Exploring Your Options, a narrative-based responsive video developed to support young female patients in understanding and negotiating the fertility preservation options available to them following a cancer diagnosis. By reflecting on findings resulting from the process of designing and analysing Exploring Your Options with the involvement of cancer-experienced young people, health professionals and related stakeholders, we reveal opportunities for designing health information videos leveraging personalisation and interactivity to meet both informational and emotional support needs of patients, and highlight insights and considerations that can guide designers seeking to leverage these opportunities in ways that are sensitive and appropriate to the demands of complex healthcare contexts.

We present a virtual reality (VR) experience designed to raise awareness towards the impact of low-vision conditions on social interactions for patients. Specifically, we look at age-related macular degeneration (AMD) that results in the loss of central visual field acuity (a.k.a. a scotoma), which hinders AMD patients from perceiving facial expressions and gestures, and can bring about awkward interactions, misunderstandings, and feelings of isolation. Using VR, we co-designed an experience composed of four scenes from the life of AMD patients through structured interviews with the patients and orthoptists. The experience takes the perspective of a patient, and throughout the scenarios, provides voiceovers on their feelings, the challenges they face, how they adapt to their situation, and also bits of advice on how their quality of life was improved through considerate actions from people in their social circles. A virtual scotoma is designed to follow the gaze of the user using the HTC Vive Focus 3 headset with an eye-tracking module. Setting out from a formal definition of awareness, we evaluate our experience on three components of awareness – knowledge, engagement, and empathy – through established questionnaires, continuous measures of gaze and skin conductance, and qualitative feedback. Carrying out a experiment with 29 participants, we found not only that our experience had a positive and strong impact on the awareness of participants towards AMD, but also that the scotoma and events had observable influences on gaze activity and emotions. We believe this work outlines the advantages of immersive technologies for public awareness towards conditions such as AMD, and opens avenues to conducting studies with fine-grained, multimodal analysis of user behaviour for designing more engaging experiences.

Social interactions hold immense significance in the lives of individuals. Children with a diagnosis of autism spectrum disorder (ASD) may face difficulties when engaging in social situations. These challenges often manifest as neurodivergent behaviours and a reduced willingness to engage in social sharing. Sharing is an integral aspect of social interaction. A lack of awareness or a limited inclination to share can hinder the acquisition of vital social skills. In this paper, a virtual reality (VR) application inspired by the Social Story™ intervention is presented. The aim of the work is to help children with ASD understand social scenarios and gain sharing skills. The application can be used in two configurations: 2D screen and head-mounted display (HMD). Since the system is being designed for a vulnerable group, as part of the iterative application design process, we solicited opinions from six expert professionals who work with children with ASD. They tested, evaluated (questionnaires and physiological data capture) and provided open feedback on the system. The key recommendation from the expert assessors is that the HMD-based interventions are more suitable as an advanced teaching tool for showing skills and knowledge, whereas screen-based interventions could be used for daily practice in children with ASD. Experts evaluated the two configurations from three factors: immersion, enjoyment, and comprehension. In terms of physiological responses, there was a greater increase in electrodermal activity (EDA) responses for the HMD-based intervention compared to the 2D intervention. The significant potential of such a system was highlighted by the expert users.

This paper presents the resources utilized for and gathered from a subjective QoE experiment conducted with 45 participants who watched 360° videos processed with offline alignment edits. The aim of these edits is to redirect the assumed field of view to a specified region of interest by rotating the 360° video around the horizon. The user experiment involved alignment edits employing both gradual and instant rotation. We employed the Double Stimulus method, whereby participants evaluated each original-processed video pair, resulting in a dataset with 5400 comfort and sense of presence ratings. During video consumption, head motion was recorded from the Meta Quest 2 device. The resulting dataset, containing the original and processed videos, is made publicly accessible. The accompanying web application, developed for the execution of the experiment, is released alongside scripts for the evaluation of head rotation data in a public repository. A cross-analysis of QoE and HM behavior provides insight into the efficacy of alignment edits for attention alignment within the same scene. This comprehensive set of experimental resources establishes a foundation for further research of 360° videos processed with alignment edits.

With the popularization of the metaverse, virtual reality mapping technology based on digital twins has generated a large amount of spatial data. These data are multidimensional, multi-scale, mobile, and distributed. In order to fully utilize these data, we propose a non mutation multidimensional hash geocoding that can organize and store data with geographic features, and achieve data mapping at different scales from macro to micro. The mapping between them can achieve joint utilization of data of various scales. On this basis, we propose a block network secure storage mapping model for spatial digital twins, which can securely and reliably organize and map spatial data. This article also looks forward to the possible emergence of digital twins of different dimensions and scales in the future metaverse, and proposes an adaptive 3D reconstruction method based on this to adapt to digital twins models of different scales in the metaverse. On the basis of our work, we will further promote the development of the spatial digital twin metaverse.

Stereoscopic 360° panoramas are a popular modality for creating cinematic virtual reality experiences. However, media in this format are typically static entities that consumers passively view. This is because objects visible in the scene, and camera properties such as depth of field, must be determined at capture time. We propose a real-time technique for dynamically synthesizing stereoscopic 360° panoramas from a collection of depth enhanced monoscopic panoramas. Using depth information, pixels can be transformed into three-dimensional space and re-projected to a different camera location. Our technique allows for head-motion parallax, dynamic depth of field, and integration of properly occluded virtual objects into the captured scene. Our technique shows minimal discrepancies compared to ground truth captures and reduces error when compared to existing stereoscopic 360° panoramic synthesis techniques. Additionally, our technique makes creating interactive extended reality experiences more accessible since monoscopic 360° cameras are much more common than their stereoscopic counterparts.

Advancements in Natural Language Processing (NLP) and Computer Vision (CV) are revolutionizing how we experience sports broadcasting. Traditionally, sports commentary has played a crucial role in enhancing viewer understanding and engagement with live games. Yet, the prospects of automated commentary, especially in light of these technological advancements and their impact on viewers’ experience, remain largely unexplored. This paper elaborates upon an innovative automated commentary system that integrates NLP and CV to provide a multimodal experience, combining auditory feedback through text-to-speech and visual cues, known as italicizing, for real-time in-game commentary. The system supports color commentary, which aims to inform the viewer of information surrounding the game by pulling additional content from a database. Moreover, it also supports play-by-play commentary covering in-game developments derived from an event system based on CV. As the system reinvents the role of commentary in sports video, we must consider the design and implications of multimodal artificial commentators. A focused user study with eight participants aimed at understanding the design implications of such multimodal artificial commentators reveals critical insights. Key findings emphasize the importance of language precision, content relevance, and delivery style in automated commentary, underscoring the necessity for personalization to meet diverse viewer preferences. Our results validate the potential value and effectiveness of multimodal feedback and derive design considerations, particularly in personalizing content to revolutionize the role of commentary in sports broadcasts.

Esports - competitive video games watched by online audiences - are the fastest growing form of mainstream entertainment. Esports coverage is predominantly delivered via online video streaming platforms which include interactive elements. However, there is limited understanding of how audiences engage with such interactive content. This paper presents a large-scale case study of an interactive data-driven streaming extension developed for Dota 2, reaching over 300,000 people during the DreamLeague Season 15 DPC Western Europe tournament. The extension provides interactive live statistics, analysis and highlights reels of ongoing matches. This paper presents an analysis of audience telemetry collected over the course of the four week tournament, introducing a novel approach to analysing usage data delivered seamlessly in conjunction to a linear broadcast feed. The work presented advances our general understanding of the evolving consumption patterns in esports, and leverages esports as a lens to understand future challenges and opportunities in interactive viewing across sports and entertainment.

Esports has become a popular field of research, enabling advances in areas such as machine learning and environment modeling. However, complex modeling systems require complex visualisations. Despite that, visualisation of complex modeling systems within esports have been limited or fragmented, particularly when focused on the audience. Furthermore, the use of data visualisation and data-driven storytelling has been proven to be an effective and imperative method for enhancing audience experience for esport spectators. Therefore, this paper investigates data visualisation techniques within esports, and compiles design considerations for developing visualisation tools for esports broadcast. This is achieved through a case-study, in which the WARDS model was utilised in live coverage of a Dota 2 tournament and evaluated through observational data.

Podcasts present challenges in information retrieval due to their non-visual nature and extended length. To understand these challenges, we conducted interviews with 12 podcast users and identified difficulties in grasping the overall podcast content with metadata alone, highlighting the necessity of navigating to specific segments. Based on this finding, we propose a browsing method that utilizes Large Language Models (LLMs) and image generation models to segment podcast contents, integrating visual cues for supporting efficient navigation. To investigate how this new method differs from conventional approaches and to evaluate its effectiveness, we conducted another user study with 12 participants. The results revealed that keyword search is ineffective when dealing with unfamiliar or inaccurate keywords. Additionally, it requires thorough examination of the script to comprehend the overall content of each episode. On the other hand, segmenting the contents and labeling the topic for each segment facilitated was found to be helpful for understanding of the overall content, enabling easy navigation to desired topics. Furthermore, we found that providing an image enabled participants to easily distinguish one segment from another, which was preferred by participants. This multimodal browsing approach is expected to establish a foundational framework for the effective browsing and comprehension of audio content, extending its applicability beyond podcasts to various forms of audio files.

Advances in generative text-to-image models are enabling new forms of personalized and adaptive media. We investigate the potential of such techniques through a generative adaptation of the fairy tale of Little Red Riding Hood. Specifically, we test two kinds of adaptations: (1) continuously adapting the visuals based on a face-to-emotion model, and (2) eliciting viewers’ fears and adapting the story accordingly. In either case, the adaptive versions are designed to make the story more scary and thus enhance the viewing experience in this dimension. We compare both variants against a baseline condition in a between-subjects study with 97 participants. Our results show that these adaptations significantly alter the viewing experience, modulated by viewers’ genre preferences.

The integration of artificial intelligence (AI) into creative domains is increasing, presenting both challenges and opportunities. In screenwriting, personal artistic expression is a fundamental aspect of the creator’s identity and work. The current use of AI in such creative processes can sometimes overshadow the creator’s vision and lead to a reduced sense of ownership over the final product. We introduce wr-AI-ter, an interactive application consisting of four basic stages: Ideation, Structure, Refinement, and Export. While some related work focuses on experts The application is intended to aid users with varying levels of screenwriting proficiency in generating screenplays using artificial intelligence, while preserving their sense of authorship. We conducted a user study with 23 participants, who had different expertise (screenwriting, documentary filmmaking, and VFX artistry). The results indicate that AI has the potential to accelerate the screenwriting process and improve the quality of scripts without compromising the sense of ownership.

In the current media landscape, understanding the framing of information is crucial for critical consumption and informed decision making. Framing analysis is a valuable tool for identifying the underlying perspectives used to present information, and has been applied to a variety of media formats, including television programs. However, manual analysis of framing can be time-consuming and labor-intensive. This is where large language models (LLMs) can play a key role. In this paper, we propose a novel approach to use prompt-engineering to identify the framing of spoken content in television programs. Our findings indicate that prompt-engineering LLMs can be used as a support tool to identify frames, with agreement rates between human and machine reaching up to 43%. As LLMs are still under development, we believe that our approach has the potential to be refined and further improved. The potential of this technology for interactive media applications is vast, including the development of support tools for journalists, educational resources for students of journalism learning about framing and related concepts, and interactive media experiences for audiences.

We report results from a systematic literature review on the topic of reward systems for older people engaging with serious games, involving an analysis of 121 scientific articles identified from multiple bibliographic databases. Our analysis, employing various taxonomies of game type, genre and challenges, reward types and features, and interaction devices and feedback modalities, reveals preponderant attention on visual feedback, sports games, and time pressure/reaction challenges, but limited scientific research on unlocking mechanisms, achievement, item-granting rewards, haptic feedback modalities, or multi-player scenarios in serious games designed for older adults. These findings reveal reward mechanisms as a distinct type of interactive media featuring some, but ignoring many design possibilities. Based on our findings, we propose several recommendations for future research on designing reward mechanisms in serious games that adapt to users’ varying abilities, favor a diversity of input and output modalities, and expand to multi-player and multi-genre setups.

Virtual reality headsets enabling immersive experiences have been used for a long time in various public spaces such as universities, vocational education, and libraries. There is a plethora of different content available for VR headsets that can support education or entertainment, such as games and movies that enrich users' lives. However, there is little literature on the use of VR games by older adults. In addition, the understanding of how older adults experience virtual worlds and games is still limited. It is believed that they experience barriers with gaming and may consider it a waste of time. Older adults, however, can become active gamers if they are offered adequate support measures and when age-related factors are taken into account in the design of the games. Video games can enrich the lives of older adults and aid in managing aging challenges. Thus, it is important to study older adults' playful experiences with VR games. This study examines older adults' playful experiences of VR games and how they experience a VR gaming session in general. Through field observations collected from 33 participants during seven gaming sessions, the results point to several issues related to enjoyable, meaningful, and immersive game experiences: Confusion, fear, thrill, excitement, interest, curiosity, and enjoyment were the most common playful experiences in these VR games. The study revealed that in some cases, older players experienced fear of falling due to the content of the VR world. Most of the participants in the study were excited and some even wanted to continue using the VR glasses after the field experiment ended. This highlights the potential of VR gaming, necessitating further research into the playful experiences of VR gaming with older adults.

When learning programming concepts, beginners face challenges that lead to decreased motivation, Game-Based Learning (GBL) uses game principles to make learning more engaging, and VR has been explored as a way to enhance GBL further. This paper explores the impact of Virtual Reality (VR) on learning programming, and we developed Codeseum to compare whether VR-based learning is perceived as more engaging and usable than a desktop game counterpart. The experiment with ten participants included data from questionnaires, interviews, and structured observations. The quantitative analysis indicated that VR was perceived as inducing higher focused attention, aesthetic appeal, and reward, while the thematic analysis provided discussion elements of seven themes, including interaction, engagement, and physical expressions from the participants. Overall, the desktop application had better accessibility, whereas the immersive interactions from Codeseum in VR induced higher levels of enjoyment and engagement. Our study contributes insights into the potential of VR in education, mainly teaching coding skills in engaging ways, and offers information to adopt immersive technologies in teaching practice.

Sedentary occupations and recreational activities carried out primarily while seated promote extended time periods spent in unhealthy sitting postures, contributing to physical and mental health issues. While apps and reminders can be effective, they often fail to sustain enjoyment and motivation or do not target stationary settings. In our work, we investigate whether sedentary waiting periods could be broken up through gentle full-body movements via full-body interactions on a large touchwall instead of remaining seated or standing. In a mixed-methods study (N=18), we compared a Match-3 game played (1) on a full-body touchwall, (2) on a smartphone standing, and (3) on a smartphone sitting, investigating user experience, performance, and acceptance. The touchwall game subtly motivated people to move, stretch and bend their bodies without performance loss while enjoying the game compared to the smartphone conditions. We suggest that full-body touchwall interaction has the potential to fill occasional waiting time while encouraging breaking up sedentary behavior.

Head-mounted displays (HMDs) offer new possibilities for reclaiming and improving passenger travel time. Common Virtual Reality (VR) HMDs provide an immersive mobile media viewing experience, overcoming the screen size constraints of mobile and seat-back displays. However, they also reduce environmental awareness as they completely occlude reality. Conversely, Augmented Reality (AR) glasses afford environmental awareness by default, allowing unrestricted presentation of media through large virtual planar displays, but without the same immersion as VR. Our work transposes the immersive benefits of VR to AR, whilst preserving awareness of the environment. We present results from a user study (N=24) exploring the addition of plants and textures as augmentations of a simulated VR train environment to enhance passenger media immersion when watching a nature documentary. We evaluated two different fields of view (50°, 104°) and three different levels of ARTV ornamentation (from none to a fully textured jungle environment in the train). We found that on high FOV devices, ARTV ornamentation results in levels of media immersion that are not significantly lower than those achieved with full VR whilst retaining environmental awareness not possible with typical VR solutions. On low FOV devices, ARTV ornamentation loses effectiveness and distract users. Our results show that the immersiveness of AR can be significantly increased while still allowing the awareness of the environment that travellers need to manage their journeys.

Television has long since been a uni-directional medium. However, when TV is used for educational purposes, like in edutainment shows, interactivity could enhance the learning benefit for the viewer. In recent years, AR has been increasingly explored in HCI research to enable interaction among viewers as well as viewers and hosts. Yet, how to implement this collaborative AR (CoAR) experience remains an open research question. This paper explores four approaches to asynchronous collaboration based on the Cognitive Apprenticeship Model: scaffolding, coaching, modeling, and collaborating. We developed a pilot show for a fictional edutainment series and evaluated the concept with two TV experts. In a wizard-of-oz study, we test our AR prototype with eight users and evaluate the perception of the four collaboration styles. The AR-enhanced edutainment concept was well-received by the participants, and the coaching collaboration style was perceived as favorable and could possibly be combined with the modeling style.

The use of Pervasive Augmented Reality (AR) in industrial settings, particularly in the context of Industry 4.0, is a growing area of research. Industry 4.0 is characterized by the integration of advanced technologies, including AR, to enable smart, connected, and autonomous systems in manufacturing and other industries. This study focuses on two systems that use Pervasive AR for logistics and data monitoring on assembly lines. Such systems were designed using cooperative Human-Centered Design (HCD) methodologies in collaboration with partners from the industry sector. We report lessons learned and guidelines on the development of AR systems for industrial environments. In addition, a user study based on a data monitoring laboratory realistic procedure with 8 participants is described. The study compares the usability and effectiveness of a dashboard on Pervasive AR and a web tablet. Overall, results emphasize that AR was better suited when requiring spatial awareness and interventions, while traditional web tablet was better for tasks that require more interaction with the content and faster data monitoring. These findings had implications for the design of industrial tools, such as prototype improvements and idealize new methods, which can guide future efforts to develop and deploy AR applications.

The concepts of sacred places and spaces appear throughout religions globally. Places such as churches, cathedrals, temples, mosques, synagogues and graveyards are given special meanings, both functionally and spiritually, and separated from the ordinary. Recently location-based augmented reality (AR) technologies and applications have become widespread, and this raises questions regarding how AR content relates to sacred places. In this study, we approached this complex topic by asking clergy of the Lutheran Christian Church (N=47) to reflect on associated phenomena. We approached the data via reflexive thematic analysis and uncovered tensions related to (1) connected vs detached from sacredness; (2) supporting the spiritual purpose of the space vs conflicting with it; and (3) maintaining tradition vs embracing innovation. Overall, our findings suggest that AR technologies and products impact sacred spaces on multiple levels, but currently there is no consensus among the clergy on the impact of these changes.

Mixed Reality (MR) gradually integrates into public consciousness, accelerating the need to develop commercial MR applications. However, the prohibitive cost of devices, the complexity of modelling software, the learning difficulties with programming languages, and the need for fidelity and interaction pose significant preliminary obstacles for developers and designers. Therefore, this paper intends to resolve the prototype obstacles in the early to mid stages by designing a rapid paper prototyping tool suitable for multi-disciplinary designers to participate in developing mixed reality applications and environments. The MR paper prototyping (MRPaperPro) technique attempts to contribute to developing MR content that mediates between information-orientated and entertainment-orientated approaches by developing affordable, readily accessible, medium-high visual and interaction fidelity tools. It requires the construction of multilayer paper modules, repositionable model components, and spectacles that analogue the HMD's field of view to facilitate designers to interact with users directly and capture manifold responses. Finally, it elaborates on the findings of the experiments.

Personal audio technologies empower individuals to manage their own auditory environment, affording control over both digital and real-world sounds. However, this control comes at a potential cost to social interaction, awareness of surroundings, and safety. This paper explores and investigates leveraging personal audio technologies as mediators in establishing personal audio spaces in homes. Our research methodology consisted of two key phases. Initially, a series of design workshops involving 16 participants were conducted to explore potential enhancements to everyday listening situations through technology. Subsequently, a within-subjects mixed-method study with 18 participants was undertaken to investigate the user experience of employing personal audio technologies to create a personalised audio space. The findings provide insights into future opportunities for listening practices, audio content, interpersonal listening, enhancing listening experiences as well as experiential components. Drawing from our results, we engage in a discussion on the potential manifestations of personal audio spaces, while also addressing current technological limitations and suggesting avenues for future research directions.

Text input in MR(Mixed Reality) provides options for users to model in details instead of just placing objects, however, 3D modeling with text input costs computation and takes time. To overcome this hurdle, we let the text-image model judge 3D layout of furniture parts. Since vanilla text-image model can not judge furniture stability, we tested two approaches: 1. combine with geometric loss, and 2. fine-tuning the model. We report the comparison of these two approaches and discuss further development for MR integration of our system.

Missing and sparse data and the associated uncertainty are inevitable in science, and their accurate portrayal in media is essential for upholding scientific transparency and credibility. Yet, in an era of conflicting information and deceptive sources, revealing uncertainty can be seen as unfavorable, in particular for science engagement media. Our study focused on conveying incomplete data on Venus’s upper atmosphere to an adolescent audience using a scientific visualization designed for a planetarium’s dome. Through a comparative study of visualizations with an unprocessed versus a processed dataset, we found that translating data into a Voronoi diagram can make the concept of sparse data understandable and aesthetically pleasing to a broader audience, yet it can come at costs of lower perceived details and accuracy. Additional results hint that participant’s preference for visualization can differ from their perceptions of clarity and that neither the preference nor the clarity appears to be linked to participants’ science literacy. Finally, we discuss the potential to design immersive science media experiences in planetariums and dome settings.

Extended Reality (XR) has emerged as a transformative and immersive technology with versatile applications in content creation and consumption. As XR gains popularity, companies eager to adopt it often possess a surface-level understanding, investing significant resources without effectively addressing the genuine needs of end-users. This study explores the current workflows of XR production companies, and the potential of social XR in mitigating challenges throughout the XR production workflow. We present the outcomes of three respective focus group workshops conducted with three XR production companies and their experts (N=17). The results indicate that at every stage of the production, namely pre-production, production, post-production, and post-release, there are communication challenges between producers and clients, as well as different production and post-production specialists. We discuss various aspects of XR concerning the problem and propose novel opportunities offered by social XR to ameliorate those challenges, improving communication and making development more agile.

Environments can affect mood, motivation, and productivity. Green spaces, for example, are known to have calming effects on people’s moods. In virtual reality (VR), we could take advantage of these effects, as we have full visual control over the environment. In this paper, we explore how such potential effects caused by the environment impact performance and user experience (UX) when playing an exergame. We created four environments differing in their level of detail and visual realism.: (1) a white room, (2) outer space, (3) an abstract space, and (4) a forest environment. In a user study (N=26) in which participants played an exergame in all four environments, we found evidence that VEs influence enjoyment and performance. The simulation of green spaces or abstract VEs with enjoyable background sounds has a particularly positive impact. We discuss how environmental features impact performance and UX and present promising avenues for future work investigating specific parts of environmental features.

Immersive media is an advanced form of applications such as Virtual Reality (VR), Augmented Reality (AR), and eXtended Reality (XR). It allows users to move freely in a video space that makes use of the real world and virtual world without distinction, and to view images from any location. It is a 6 degrees of freedom (DoF) media that provides users with experiences that feel as though they are in the video space. We are currently promoting research and development to realize immersive media that can be used on various types of presentation devices. If immersive media can be accessed from existing devices such as smartphones and PCs, as well as from devices that provide a particularly realistic feeling, such as head-mounted displays, users will be able to utilize immersive media on more occasions. Moreover, it is desirable for content creators that one immersive content be usable on various types of presentation devices, rather than each immersive content having to be newly created for a specific type of device. We consider immersive content to be content that does not assume specific device capabilities. In this work, we propose a system that renders the content on a cloud in order to produce images for presenting on devices having different capabilities. This paper reports our experimental finding confirming immersive content consisting of scene descriptions and volumetric video can be presented on three types of presentation devices using a renderer built on a virtual server on a cloud.

As novel media experiences turn towards audience personal data for enhancement, a range of socio-technical challenges that confront their adoption into the everyday lives of audiences are simultaneously surfaced. Podcasts, being the latest member of this emergent phenomena help present the next generation of challenges here like the management of trust within scenarios of implicit (audio-based experiences’) data collection and the consideration for real world social scenarios where these media experiences are shared. This paper reports the conceptualisation and design phase of the first adaptive podcast that seeks to explore these questions of management of trust and sociality of data through the application of the principles of Human Data Interaction, particularly the provision of scope for user data negotiations.

• Human-centered computing;

Airport security screening processes in the USA can be intimidating and opaque for passengers. As a result, passengers tend to wait for direct instruction and take a passive role. This can impact the efficiency of the system and slow throughput, contributing to the very congestion and delays that cause passengers’ frustration in the first place. However, we may be able to avoid this stress on the security screening processes and improve passenger well-being overall with the introduction of an autonomous self-screening checkpoint design. As an interdisciplinary team, we are developing a self-screening experience that is integrated across digital, physical, and human interfaces creating seamless and passenger-focused interaction. For the purposes of this paper, we provide an example of how our passenger-focused strategy is embodied in our passenger-facing graphical user interface and invite discussion about how this can improve passenger interactions with media in a way that is context-aware.

This work-in-progress paper delves into the effects of AR beauty filters on social media, scrutinising their contribution to unrealistic beauty standards and their psychological impact on users. Utilising objectification theory, it explores the reinforcement of gender biases through these filters. The study highlights gender disparities in TikTok filter usage and suggests a transparency-enhancing "disclaimer" feature. It further addresses the regulatory implications of the AI Act on the development and use of AR filters. Ultimately, the study calls for augmented transparency and regulatory oversight to address the broader societal implications of beauty filters and their perpetuation of biased stereotypes.

This paper describes the Telemersive Toolkit [hereafter referred to as TTkit], a system that enables artists and educators to set up complex low-latency multimedia streaming infrastructures between multiple computers connected via networks. TTkit addresses the challenge for non-technical personnel to create their own streaming environment that does not require reconfiguration of the local network infrastructure and allows computers to be connected across different network configurations and firewall setups. The TTkit provides an intuitive user interface called telemersive-gateway [hereafter referred to as Gateway] to set up the network and displays the current status of all streams – video, audio, motion tracking data, control data – exchanged within the network. Each Gateway on the network can monitor and configure the streaming setup of all other Gateways on the network, making it a very convenient tool for troubleshooting and helping artists, researchers as well as educators (e.g. to assist novice users) in setting up their individual machines. In order to evaluate TTkit, the Telematic Perfomance Format Research Group has used it for its three most recent project realisations in the form of telematic multimedia music and theater performances, one of which, called ’Osmosis’, is described in this paper to give a clearer understanding of the practical application of this system. Our findings suggest that TTkit is ready to be used by a wider community, and we believe that it will help artists and educators to create their own networked performance environments without spending too much time setting up the technical infrastructure, and thus focus more on the artistic freedom that the new system offers.

How can interaction design counter the trend of declining cinema visits by improving the visitor experience? This project focuses on experiences of social immersion and shared emotions among cinema audiences. The After-movie Social Accelerator is based on specific key design criteria for emotion-sharing and offers a proof-of-concept implementation. The paper closes with a preliminary evaluation (n=6) that indicates the value of emotion-sharing in cinema experiences but also calls for more communication opportunities.

Live streaming with mobile phones is a common practice where streamers and viewers use various resources for interaction. Based on the method of multimodal conversation analysis, we examine recordings of a clay sculptor's live streams on a Chinese social app. We address how the streamer's dual involvements—doing the sculpture work and responding to viewers’ messages—are achieved moment-by-moment. We will demonstrate how the streamer uses multiple resources, such as language, body torque, facial expressions, eye gaze, phone adjustment, and the “disrupted turn adjacency” feature of viewers’ messages, to achieve multiactivity by holding two intersecting courses of action, and how he may use live streaming to achieve self-exposure, chat, and virtual intimacy during his routine work.

This paper explores how the Socratic method, implemented in an AI-chatbot, can be used to stimulate citizens’ critical thinking, and consequently fight disinformation. In the Horizon Europe project TITAN, we are scrutinizing this opportunity. A prototype of the Socratic AI-chatbot was tested in four Co-Creation Labs with citizens. The findings presented in this paper indicate that there is potential and provide guidance to improve the effectiveness of the Socratic AI-chatbot in stimulating critical thinking and recognizing disinformation.

Given the profound impact of staying socially connected on the well-being of older adults, this study explores the potential of augmented reality (AR) systems to enrich their social lives. A wearable AR communication system prototype was developed and tested in a user study involving N = 16 older adults from Germany. Participants wore an AR headset and engaged in a conversation task with a remote person represented by an avatar. Older adults’ experiences were assessed using think-aloud protocols, qualitative observations, posttest questionnaires, and semi-structured oral interviews. Preliminary findings indicate overall participant satisfaction, with minimal observed difficulties in headset usage and avatar-mediated interpersonal communication. The positive engagement during AR conversations highlights the system’s potential to provide positive communication experiences among older individuals. This work-in-progress paper introduces the developed system prototype and outlines the conducted user study. Further data analyses will provide deeper insights into older adults’ experiences with the system. The results will contribute to refining the prototype and offer valuable insights for the development of AR communication systems tailored to the needs and preferences of older adults.

For many individuals, greeting someone with a simple 'hello' is a common occurrence that can be accomplished quickly using a mobile phone. But what about older people who haven't mastered this type of resource? When they miss someone, how do they send a message like "good morning, I miss you"? It's something that might not be done if they don't have access to the necessary resources. This paper aims to explore technological empowerment for older adults by combining TV messages and voice commands. 110 people aged between 59 and 91 were interviewed to give their opinions on these two mechanisms. The main conclusions are that there is no consensus on notifications, but there is strong interest in the subject, particularly as a very familiar device is used. Voice features were fully accepted because they allow for more natural interactions, without the need for physical contact with a device. The combination of both features is another point that arouses curiosity in the group, whose opinions are described in detail throughout the paper.

Modern in-vehicle infotainment systems connect to drivers’ personal digital devices, such as smartphones, to facilitate calls, notifications delivery, and access to online media and resources while driving. This communications process facilitates distributed user interfaces (DUIs) and allows casual in-vehicle interactions. However, such functionalities of modern vehicles still lack the extensive capabilities of state-of-the-art DUIs, which can offer a variety of interaction opportunities through features such as migration, redirection, adaptation, and granularity, among others. In this work, we explore DUIs in the specific context of smart vehicles, and introduce V2DUIs (Vehicle-to-DUIs) as a new addition to the V2X (Vehicle-to-Everything) family of smart vehicle concepts.

This study introduces a cross-modal, tongue-involved interface designed to enrich taste and touch perception within Mixed Reality (MR) environments. It features a novel gear-like mechanism loaded with various flavored conductive jellies, forming a cross-modal platform. This interface enables the gear to present different flavor units to the user’s tongue, activated by specific virtual events. This setup fosters mutual communication and a playful experience between the physical and virtual worlds, leveraging the synesthesia of taste and tactile perceptions.

The current state-of-the-art 2D data visualizations fall short in capturing the intricate complexity and depth of available information crucial for integrated decision-making. In response to this limitation, the Systems Exploration and Engagement environment (SEEe) emerges as a cutting-edge virtual immersive analytics data experience. We developed this system through a user-centered design process involving an interdisciplinary design and development team. Through virtual reality, SEEe seamlessly integrates geo-referenced spatial data, abstract data visualization, and qualitative data encompassing text, images, videos, and conceptual diagrams to support sensemaking from large amounts of multiformat data and integrated decision making. We aim to redefine the experience of analyzing extensive amounts of abstract data by creating an environment that accommodates both quantitative and qualitative data for visualization and analysis. How these novel immersive analytics experiences fit into data analysis workflows in various domains have not been studied widely. We carried out a user study with 10 public health graduate students to test the usability, learnability, and utility of the SEEe experience and to explore how these immersive data visualization experiences can fit into traditional data analysis processes. While SEEe is designed to be adaptable across various domains, we evaluated its performance within the public health context. The results of the evaluation affirm that SEEe is not only usable and useful but also provides a learnable environment conducive to immersive analytics.

Emergency Medical Services (EMS) professionals undergo continuous training, which is crucial for handling high-pressure situations. Considering the time constraint these professionals are under, innovative approaches are necessary to enhance the effectiveness of EMS training. Probabilistic interactive video training is a promising avenue. It employs interactive web-based platforms to create immersive learning experiences via a standard web browser. Personalised training using non-markovian processes tailors the training to individual trainees’ needs and enhances engagement. The created probabilistic models simulate realistic emergency scenarios that foster the development of robust decision-making skills under uncertain and time-critical conditions. The research aims to analyse the effectiveness of interactive video-based (regular and 360-degree) training and explore its potential as an innovative approach to enhance EMS training using the Technology Acceptance Model (TAM) and interviews with police and ambulance students.

Financial literacy is essential for personal and national economic well-being, demanding attention in today’s dynamic society. However, a concerning trend persists where individuals may lack basic financial knowledge; this is common even among individuals with a high level of education. This gap is especially concerning for young adults, as it prevents them from making informed decisions, managing finances effectively, and securing a stable future. This paper presents the development and pilot evaluation of a serious puzzle game prototype to enhance financial literacy among young adults. The pilot evaluation (N=7) explores various game mechanics to deliver engaging financial education. Initial findings indicate promising potential for serious puzzle games in facilitating financial literacy. The research aims to contribute insights into pedagogical strategies and game design enhancements, fostering informed decision-making and economic stability among young adults.

Video games have become an integral part of people’s lives. However, visually impaired players are usually unable to navigate current games. The goal of this research is to help visually impaired people navigate 3D games while also maintaining immersion. In the future, we hope to create a plug-in for game engines such as Unreal Engine and Unity that will be used by the game industry to easily make their game accessible to visually impaired players. This demo shows an early iteration of this concept. The goal is to gather input from the interactive media experience community on how to proceed with this process.

Effectively communicating vital sustainability topics, such as personal CO2e emissions stemming from daily activities, is challenging since there is a general lack of "carbon literacy." Despite efforts, like employing data visualizations, conveying this information in engaging and memorable ways remains difficult, often resulting in rapid forgetfulness. Our demonstration presents a pedagogical approach aimed at addressing this challenge. Through the integration of surprise elements, data physicalization, and embodied learning, we have developed two interactive mediated non-digital learning experiences focusing on CO2e emissions from food and from consumption. At the core of our pedagogical approach are physical artifacts representing CO2e emissions where the weight of the artifacts directly corresponds to the CO2e emissions they represent. By employing a structured interactive pedagogical setup involving peer discussion, hands-on manipulation of the artifacts, and subsequent explanations, we have created an engaging experience that encourages long-term learning.

Cognitive Behavioral Therapy is a empirically validated treatment for anxiety and depression. By identifying negative thinking patterns (cognitive distortions), individuals are able to reevaluate and reframe thinking patterns for better mental health. Building on previous research in digital health interventions, we explore the use of games and VR for increasing awareness of cognitive distortions. CBT-Darts is an immersive VR game where players match statements to cognitive distortions by throwing darts at balloons. By mixing a physical and cognitive task, we hope to create an engaging and fun way to teach how to identify cognitive distortions.

In this demonstration, we present a system for creating and viewing spatial memories using mobile devices. The concept of interacting with three-dimensional recordings of people and places has been explored using depth cameras and Augmented and Virtual Reality headsets, but these methods require specialized hardware that has not yet reached wider availability. Recent developments in mobile camera technology have presented new capabilities which we leverage in our application for Spatial Mobile Memories. The application captures visual and depth information, together with the orientation and movement of the mobile device. This is then combined into a single standard image or video file. Users can share these files via personal messaging channels, and replay them in 2D and 3D across devices using the same application. The goal of this project is to develop open and widely-accessible systems for the creation and viewing of spatial memories that leverage the ubiquity and familiarity of mobile devices.

Acquiring depth data has become easily achievable with advancements in depth sensing and depth estimation technologies. As a result, obtaining a depth stream to describe the topology of a corresponding video stream has been considerably simplified. Presence of a depth stream offers numerous benefits, including the integration of advanced visual enhancements to the corresponding video stream in a flexible and efficient manner. This can enrich video-centric applications and facilitate their transition to Augmented Reality (AR) environments, where processing capabilities and battery power are limited. In this paper, we introduce VidDepth, an application developed for mobile devices to demonstrate examples of visual enhancements in video playback scenarios across both traditional and AR settings.

This demo presents an interactive communication system based on immersive media for analyzing the users’ Quality of Experience in collaborative tasks. The interaction between users is studied in an asymmetric scenario where a peer-to-peer communication has been set up between a PC and a Virtual Reality headset. Two application scenarios have been considered: a Block Building task and a Treasure Hunt game. The two users will cooperate to perform the two tasks. The goal is to study the relation between the type of transmitted information (i.e., audio and video or audio only) and the quality and quantity of interaction. During the demo, participants will have the opportunity to try one of the designed applications.

We introduce ChairMX, on-chair input technology for novel interactive media consumption experiences through the chair. ChairMX is the end result of an elaborate four-stage research and development process involving a systematic literature review of the extent in which chairs have been used in interactive systems, involvement of potential end users to elicit their preferences for on-chair input, engineering on-chair gesture input recognition technology with wearables, and exploration of application opportunities. By leveraging a ring-like form factor wearable, ChairMX is a readily deployable technology, working for uninstrumented chairs everywhere and enabling a variety of use case scenarios involving interactive media consumption, from the living room to public environments.