"Interaction in Social Extended Reality: A Quality of Experience Approach" 

Carlos Cortés

E.T.S. Ing. Telecomunicación, Universidad Politécnica de Madrid, June 2024, "Cum Laude".

Ph.D. Thesis Advisors: Narciso García and Pablo Pérez.

The rise of immersive technologies has led to an increase in the number of use cases that adapt this type of technology within the telecommunications area. Some examples are: industrial training, multimedia content consumption and tele-training. Among all the immersive technologies, eXtended Reality through the use of Head-Mounted Displays (HMD) is the one that focuses the majority of current developments. Specifically, the Social XR paradigm frames the use of immersive technologies in a multi-user or social context. Among the decisive factors for using immersive technology in communications use cases, two stand out: the possibility of making the user believe that they has been transported to another place (sensation of presence) and the possibility of increasing interactions by allowing displacements through space (6 degrees of freedom) as well as the possibility of interacting in a more natural way. Such improvements are ultimately improvements in user experience (UX). Therefore, UX evaluation is crucial for effective XR development. In a telecommunications context, this is known as quality of experience (QoE) evaluation.

In the initial stages of the thesis development, the focus was primarily on exploring possible areas of scientific contribution. The first significant area that emerged was the proposal of a methodology for evaluating the QoE of immersive environments based on 360 video. To this end, an inter-laboratory experiment was conducted within the video quality expert group (VQEG) of the International Telecommunications Union (ITU). As a result of this experiment, the ITU-T P.919 Recommendation was published.

As the thesis progressed, another key area of exploration was the development and evaluation of natural user interfaces (NUI) in the context of industrial training. Within a public-private partnership, we developed a training environment for fiber optic review with specific object manipulation requirements. In this section of the thesis, NUI-based manipulation solutions with subjective evaluation by subject matter experts are presented. Thanks to these contributions, we have been able to confirm that such natural interfaces allow the development of training that reduce cost and environmental impact while maintaining high user satisfaction values.

As we performed interaction development for Social XR, we identified that delay appeared to be a key element in guaranteeing QoE. Therefore, the third area of scientific contribution focused on investigating the impact of latency in different processing loops within the Social XR domain. In this sense the thesis presents two major contributions, a first contribution that focuses on the study of the different delays perceptible by users and how these affect them differently. Within this same contribution, a processing framework common to different existing Social XR systems is presented. Finally, a state of the art of different studies that identify allowable latencies in different use cases involving XR communication is presented. Using these values, a QoE prediction model adapted from an ITU recommendation is presented in order to be flexible to new use cases.

The second major contribution presents three novel QoE studies investigating the impact of delays on: environment updates, self-view perception, and video conferencing within Social XR environments.
This doctoral thesis has significantly advanced our understanding of immersive video-based environments. We can now effectively assess the QoE within these environments using novel methods. Furthermore, the thesis explores the development of natural interfaces for interaction in XR, allowing us to evaluate XR interaction environments from a QoE perspective. This includes pinpointing the impact and location of delays within Social XR systems. By understanding how different delay values influence UX for various use cases, we can establish acceptable delay thresholds for optimal QoE in video-based Social XR.