Just noticeable quantization distortion modeling for perceptual video coding

Abstract

With the advent of Ultra High Definition (UHD) TVs and contents, video compression has drawn more attention than before. Because a conventional predictive video coding is reaching its limits, one of the other directions to further achieve coding efficiency is a perceptual video coding (PVC) which often exploits just noticeable distortion (JND) models for efficient perceptual redundancy reduction. Unfortunately, previous JND modeling approaches are not suitable for PVC. The conventional transform-domain JND models were designed by psychovisual experiments by increasing the magnitude of each DCT coefficient independently to model JND thresholds at a frequency. However, the magnitudes of DCT coefficients of images are generally decreased during quantization process of video compression. Thus, we presents a new DCT-based JND model by incorporating the quantization effects of video for PVC, which is called just noticeable quantization distortion (JNQD) model that estimates a JND value using a structural contrast index (SCI) in DCT domain. Our proposed JNQD model can be applied as preprocessing prior to any video compression technique by adding a parameter to adapt the model to quantization step sizes. It is called adaptive just noticeable quantization distortion (AJNQD) model. In this thesis, we propose two ways to make the AJNQD model adaptable to quantization step sizes. Firstly, the model parameters of AJNQD are determined via linear regression based on the features extracted from training block images. Secondly, we utilize a convolution neural network (CNN) as the AJNQD model adaptable to quantization step sizes. For experiments, our two AJNQD models have been applied to the state-of-the-art video coding standard, High Efficiency Video Coding (HEVC) and yielded the maximum and average bitrate reductions of 24.7% and 10.1%, respectively, compared to the HEVC random-access encoding without preprocessing without subjective video quality degradation. When the CNN regression method is applied, the preprocessing speed is about 11 times faster compared to using the linear regression method. This is because the hand-crafted feature extraction process is not required. Also, the features extracted by the CNN are more effective than the hand-crafted feature extracted for linear regression. Thus, the CNN-based AJNQD model achieved the maximum and average bitrate reductions of 53.2% and 20.8%, respectively, compared to the HEVC random-access encoding without preprocessing. The performance improvement of the CNN-based AJNQD model is about average 10.7% compared to that of the linear regression method. This performance improvement is an average of 11.2% higher than that of the state-of-the-art PVC scheme.