Creating and sharing knowledge for telecommunications

CapsField: Light Field-Based Face and Expression Recognition in the Wild Using Capsule Routing

Moghaddam, A. ; Etemad, A. ; Pereira, F. ; Correia, P.L.

IEEE Transactions on Image Processing Vol. 30, Nº -, pp. 2627 - 2642, February, 2021.

ISSN (print): 1057-7149
ISSN (online):

Journal Impact Factor: 3,315 (in 2008)

Digital Object Identifier: 10.1109/TIP.2021.3054476

Download Full text PDF ( 9 MBs)

Light field (LF) cameras provide rich spatio-angular
visual representations by sensing the visual scene from multiple
perspectives and have recently emerged as a promising technology
to boost the performance of human-machine systems such
as biometrics and affective computing. Despite the significant
success of LF representation for constrained facial image analysis,
this technology has never been used for face and expression
recognition in the wild. In this context, this paper proposes
a new deep face and expression recognition solution, called
CapsField, based on a convolutional neural network and an
additional capsule network that utilizes dynamic routing to
learn hierarchical relations between capsules. CapsField extracts
the spatial features from facial images and learns the angular
part-whole relations for a selected set of 2D sub-aperture images
rendered from each LF image. To analyze the performance
of the proposed solution in the wild, the first in the wild LF
face dataset, along with a new complementary constrained face
dataset captured from the same subjects recorded earlier have
been captured and are made available. A subset of the in the
wild dataset contains facial images with different expressions,
annotated for usage in the context of face expression recognition
tests. An extensive performance assessment study using the new
datasets has been conducted for the proposed and relevant prior
solutions, showing that the CapsField proposed solution achieves
superior performance for both face and expression recognition
tasks when compared to the state-of-the-art.