Creating and sharing knowledge for telecommunications

FACIAL EMOTION RECOGNITION USING LIGHT FIELD IMAGES WITH DEEP ATTENTION-BASED BIDIRECTIONAL LSTM

Moghaddam, A. ; Etemad, A. ; Pereira, F. ; Correia, P.L.

FACIAL EMOTION RECOGNITION USING LIGHT FIELD IMAGES WITH DEEP ATTENTION-BASED BIDIRECTIONAL LSTM, Proc IEEE International Conference on Acoustics, Speech, and Signal Processing ICASSP, Barcelona, Spain, Vol. , pp. - , May, 2020.

Digital Object Identifier:

Download Full text PDF ( 5 MBs)

Abstract
Light field cameras are able to capture the intensity of light
rays coming from multiple directions, thus representing the
visual scene from multiple viewpoints. This paper exploits
the rich spatio-angular information available in light field
images for facial emotion recognition. In this context, a
new deep network is proposed that first extracts spatial features
using a VGG16 convolutional neural network. Then, a
Bidirectional Long Short-Term Memory (Bi-LSTM) recurrent
neural network is used to learn spatio-angular features
from viewpoint feature sequences, exploring both forward
and backward angular relationships. Additionally, an attention
mechanism allows our model to selectively focus on
the most important spatio-angular features, thus enabling a
more effective learning outcome. Finally, a fusion scheme
is adopted to obtain the emotion recognition classification
results. Comprehensive experiments have been conducted
on the IST-EURECOM Light Field Face database using two
challenging evaluation protocols, showing the superiority of
our method over the state-of-the-art.