Assessment of Visual Quality and Simulator Sickness for Omnidirectional Videos (T-Labs Series in Telecommunication Services)

This book presents extensive research on the quality of 360° video perceived by users with HMDs. The book aims to develop a set of standard guidelines for the systematic visual quality assessment of 360° videos. Firstly, conventional subjective test methods such as Absolute Category Rating (ACR) and Double Stimulus Impairment Scale (DSIS) are applied to evaluate video quality, alongside the Modified ACR (M-ACR) method newly proposed. Building on the reliability and general applicability of the procedure across different tests, a methodological framework for 360° video quality assessment is then presented. The author also analyzes simulator sickness to investigate the impact of different influencing factors. The insights gained on simulator sickness related to 360° video contribute to a better understanding of this particular use case of VR and can help to improve comfort among users by suggesting improvements in the technical specifications of 360° video and HMD technology and thus improving QoE. Preface Acknowledgments Contents Acronyms 1 Introduction 1.1 Contributions 1.1.1 Visual Quality Assessment of 360 Videos 1.1.2 Simulator Sickness 1.1.3 Assessment of Simulator Sickness Questionnaire 1.1.4 Holistic Evaluation of QoE for 360 Video 1.1.5 Reproducible Research 1.2 Organization of Book References 2 State of the Art 2.1 Preliminary Concepts 2.2 360 Datasets 2.3 360 Video Quality 2.4 Tile-Based Streaming 2.5 Simulator Sickness 2.5.1 Influencing Factors 2.5.2 Evaluation of Simulator Sickness 2.6 Evaluation of Presence 2.7 Exploration Behavior References 3 Technical Setup 3.1 Overview of End-to-End Chain 3.2 Selection of 360 Videos 3.2.1 Test 1 3.2.2 Test 2 and Test 3 3.2.3 Test 4, Test 5, and Test 6 3.2.4 Test 7 3.2.5 Test 8 and Test 9 3.2.6 Test 10 3.2.7 Test 11 3.2.8 Test 12 3.2.9 Summary of Used 360 Source Videos 3.3 Generation of 360 Videos 3.4 Test Environment 3.5 AVTRack360 3.6 Conclusions References 4 Comparison of Different Subjective Test Methods 4.1 Test Design 4.1.1 Experiment 1 4.1.2 Experiment 2 4.2 Test Subjects 4.2.1 Experiment 1 4.2.2 Experiment 2 4.3 Test Methods 4.3.1 Experiment 1 4.3.2 Experiment 2 4.4 Experimental Results 4.4.1 Experiment 1 4.4.2 Experiment 2 4.5 Discussion 4.6 Conclusions References 5 Visual Quality Assessment for 360 Videos 5.1 Impact of HMDs 5.1.1 Technical Setup and Equipment 5.1.2 Test Method 5.1.3 HTC Vive vs. Oculus Rift 5.2 Encoding Settings 5.3 Tile-Based vs. Full Frame Streaming 5.3.1 Experimental Setup 5.3.2 Testbed 5.3.3 Test Setup 5.3.4 Test Method 5.3.5 Subjective Evaluation Results 5.3.6 Exploration Behavior 5.4 Impact of Voting 5.4.1 Technical Settings and Test Method 5.4.2 Results 5.5 Prediction of Video Quality Scores for 360 Video 5.6 Recommendations References 6 Simulator Sickness Assessment for 360 Videos 6.1 Impact of Technical Factors 6.1.1 Resolution 6.1.2 Bitrate 6.2 Exploration Behavior 6.3 Tile-Based Streaming 6.3.1 Influence of Content on SSQ Scores 6.3.2 Influence of Network Delay on SSQ Scores 6.3.3 Influence of Test Duration on SSQ Scores 6.4 Toward Simulator Sickness Modeling 6.4.1 SiSiMo Concept 6.4.2 Technical Setup and Equipment 6.4.3 Test Method 6.4.4 Results and Temporal Evolution Model 6.5 Conclusions References 7 Performance Analysis of SSQ 7.1 Motivation 7.2 Experiment 7.3 Simplified Simulator Sickness Questionnaire 7.3.1 Per-subject PCA 7.3.2 Exploratory Factor Analysis 7.3.3 Discussion 7.4 Conclusions References 8 Evaluation of Different Constructs: Media Quality, Simulator Sickness, and Presence 8.1 Experimental Setup 8.1.1 Technical Setup and Equipment 8.1.2 Test Method 8.2 Results 8.2.1 Audiovisual Quality 8.2.2 Evaluation of Simulator Sickness 8.2.2.1 Long Simulator Sickness Questionnaire 8.2.2.2 Short Simulator Sickness Questionnaire 8.2.3 Presence 8.2.3.1 Long Presence Questionnaire 8.2.3.2 Short Presence Questionnaire 8.2.4 Relationship Between Integral Quality, Presence, and Simulator Sickness 8.3 Conclusion References 9 Conclusion and Future Work 9.1 Summary of Results 9.2 Future Work References A List of Questionnaires A.1 Video Quality Questions A.2 Short Simulator Sickness Questions A.3 Simulator Sickness Questionnaire A.4 Presence Questionnaire A.5 Short Presence Question (Test 12) Index