VQEG Reports from Completed Validation Tests
VQEG has completed the following validation tests of objective video quality models. Each test involved a group of proponents who submitted models for evaluation. The independent lab group (ILG) implemented the test and ensured a fair and unbiased evaluation of models. The validation test completed so far examined the following types of objective video quality models:
- Full Reference (FR) directly compare the original video sequence (high quality) and the processed video sequence. These models can do pixel-by-pixel comparisons.
- Reduced Reference (RR) extract features from the original and processed video sequences, and compare those low bandwidth features.
- No Reference (NR) only have access to the processed video sequences.
- Hybrid Perceptual Bitstream (Hybrid) models have access to both the bistream and either FR, RR or NR information.
Hybrid Perceptual Bitstream (Hybrid) examined the performance of Hybrid models for VGA/WVGA and HDTV, using ACR-HR. Ten datasets were created. The test plan, final report, and status of ITU Recommendations are available on the Hybrid project page.
High Definition Television (HDTV) examined the performance of FR, RR and NR models for HDTV, using ACR-HR. Six datasets were created, five of which can be redistribted for research and development purposes. The test plan, final report, video sequences and resulting ITU Recommendations are available on the HDTV Phase I project page.
Reduced Reference / No Reference Television (RRNR-TV) examined the performance of RR and NR models on standard definition video, using ACR-HR. Two datasets were created: 525-line (NTSC) and 625-line (PAL) video. The test plan, final report, and resulting ITU Recommendations are available on the RRNR-TV project page.
Multimedia (MM) Phase I examined the performance of FR, RR and NR models for VGA, CIF and QCIF video (no audio), using ACR-HR. 41 datasets were created, none of which can be redistributed. The test plan, final report and resutling ITU Recommendations are available on the Multimedia Phase I project page.
Full Reference Television (FRTV) Phase II examined the performance of FR and NR models on standard definition video, using the DSCQS methodology. Two datasets were created: 525-line (NTSC) and 625-line (PAL) video. The test plan, final report, and resulting ITU Recommendations are available on the FR-TV Phase II project page. The video sequences cannot be distributed due to licensing restrictions.
Full Reference Television (FRTV) Phase I examined the performance of FR and NR models on standard definition video, using the DSCQS methodology. Four datasets were created: high quality and medium quality, for both 525-line (NTSC) and 625-line (PAL) video. Models were difficult to distinguish due to the narrow range of quality in each dataset. The test plan, final report, and video sequences are available on the FR-TV Phase I project project page.