Size of experiments in quality of experience and good practice for statistical significance testing

What is it about?

To perform experiments with test persons i.e. subjective experiments are usually costly and time consuming, but can give very good results if done in a good way. When planning experiments in Quality of Experience(QoE) it is good to know that at least on statistical ground that the planned number of test persons will find the an effect if it is there with a high probability. This paper will explore how to do this and give examples connected of QoE for video. It will also explore good practices for statistical significance testing both for subjective and objective QoE assessment.

Featured Image

Why is it important?

In this paper, we will investigate and discuss statistical methods used for QoE assessment, especially targeting the planning of the sample size for subjective experiments, i.e., the number of test subjects, before the actual experiment is executed. We will show that with applying traditional statistical methods, strong inferences can be obtained of the planned sample size. improve the statistical analysis per se but to demonstrate its usage and typical consequences when conducting subjective experiments for QoE analysis. Furthermore, we believe that this type of paper is lacking within the scientific domain of QoE.