Creating and sharing knowledge for telecommunications

Attention-driven tile splitting method for improved efficiency of omnidirectional versatile video coding

Carreira, J. F. M. ; Faria, S.M.M. ; Távora, L.M. ; Navarro, A. ; Assunção, P.A.

Attention-driven tile splitting method for improved efficiency of omnidirectional versatile video coding, Proc IEEE International Conf. on Image Processing - ICIP, Anchorage, United States, Vol. , pp. 2149 - 2153, October, 2021.

Digital Object Identifier: 10.1109/ICIP42928.2021.950635

Abstract
A common approach used in omnidirectional video coding is based on frame splitting into tiles, allowing partial delivery of only the subset of tiles that is necessary to render the user’s current viewing region, defined as a specific viewport or Field-of-View (FoV). Since tiles can be independently encoded, such mechanism provides a flex- ible solution for encoding planar representations with ultra-high def- inition (UHD), such as the Equirectangular Projection (ERP), using Versatile Video Coding (VVC). By only selecting and transmitting the coded data that is required to render the necessary FoV, rather than the full 360°, a great deal of bandwidth can be saved. While cur- rent solutions are based on splitting the omnidirectional video frames into tiles of equal size, this paper proposes a new approach based on adaptive tile size, driven by visual attention. Those regions where the visual attention is higher are partitioned in smaller tiles to ob- tain higher bit rate granularity, allowing to decode the most frequent FoVs with minimum out-of-FoV pixels and reduced bandwidth. Op- timal tile boundaries are found by solving a lagrangian minimisation problem with a cost function that achieves the best tradeoff between the standard deviation and the average attention-weighted bit rate per tile. The experimental results show that an average of 7.17% and 17.73% of bit rate savings is obtained in comparison with conven- tional tilling methods for the commonly used FoVs of 90° ×90° and 45° ×45°, respectively.