Institute for Telecommunication Sciences
the research laboratory of the National Telecommunications and Information Administration

Lucjan Janowski; Ludovic Malfait; Margaret H. Pinson

Abstract: The conventional video subjective test design, in which subjects view and rate multiple versions of each source video sequence, was used for decades. New technology, like adaptive streaming, makes it almost impossible to use this design since much longer sequences are needed. In this paper we examine three experiment designs: the conventional design and two alternatives that use each source sequence only once. Based on data collected by three laboratories, we compare the accuracy and scoring behavior of these three designs. We check whether there is a significant difference in scoring behavior between the experiment designs. One of the proposed experiment designs is proposed for immediate use.

Keywords: video quality; experiment design; precision; subjective test; source video; comparing tests

To request a reprint of this report, contact:

Lilli Segre, Publications Officer
Institute for Telecommunication Sciences
(303) 497-3572

For technical information concerning this report, contact:

Lucjan Janowski

Disclaimer: Certain commercial equipment, components, and software may be identified in this report to specify adequately the technical aspects of the reported results. In no case does such identification imply recommendation or endorsement by the National Telecommunications and Information Administration, nor does it imply that the equipment or software identified is necessarily the best available for the particular application or uses.

Back to Search Results