One of the major concerns for the wide adoption of 3D/omnidirectional video is the ability to provide satisfying Quality of Experience (QoE) levels to the users, notably when the video is transmitted over networks with limited bandwidth and variable channel conditions. Therefore, the development of accurate methods to assess the 3D/omnidirectional video quality, as well as techniques to ensure an efficient video transmission, is of upmost importance for content and service providers.
When compared to 2D video, the transmission of 3D and omnidirectional video poses some challenges, such as the need of characterizing the perceptual impact of the distortions and artefacts introduced by handling these video types. Naturally, the video traffic generated by 3D/omnidirectional video is several times larger than in 2D video; therefore, and noticing the popularity of mobile systems, another challenge is the design of efficient video streaming solutions.
This project addressed the challenge of enhancing the Quality of Experience (QoE) of video transmissions using wireless systems. With a special focus on 3D and omnidirectional video applications, and ranging from QoE modelling to wireless network resource allocation, an interdisciplinary approach was followed in order to provide QoE optimisation both in the wireless network and at the client side. Two new parametric models for the quality and the bit rate prediction were proposed. In addition, a new dataset of omnidirectional videos was made available to the scientific community. As for video rendering optimisation for omnidirectional images, a set of content-dependent image features was designed. Moreover, a content-dependent objective quality metric was proposed, which allows to determine the best settings in order to minimise the resulting geometric distortion impact.
With respect to 3D video, a new approach for the quality prediction of synthesized images was proposed. Furthermore, a new dataset was developed and made available to the scientific community, which comprises synthesized images containing artefacts due to texture and depth compression of the source views, as well as due to errors in the synthesis process.
Considering the wireless network resource allocation, a QoE-aware scheduling algorithm was devised, which makes use of the current buffer level at the client side and the radio channel conditions, as well as the bitrate of the requested video segment. The proposed scheduling strategy can also fulfil different satisfaction criteria, since it can be tuned to maximise the numbers of users with high QoE levels or to minimise the number of users with low QoE levels.
Project ENVISION was funded by IT and developed in a collaboration between researchers from the Radio Systems group (António Rodrigues and Ivo Sousa) and the Multimedia Signal Processing group (Paula Queluz and João Ascenso).