Creating and sharing knowledge for telecommunications

Enhancing Multimodal Silent Speech Interfaces with Feature Selection

Freitas, J.F. ; Ferreira, A. ; Figueiredo, M. A. T. ; Teixeira, A. T. ; Dias, M. D.

Enhancing Multimodal Silent Speech Interfaces with Feature Selection, Proc ISCA - Conf. of the International Speech Communication Association - INTERSPEECH, Singapore, Singapore, Vol. --, pp. 1169 - 1173, September, 2014.

Digital Object Identifier: 0

Download Full text PDF ( 177 KBs)


In research on Silent Speech Interfaces (SSI), different sources of information (modalities) have been combined, aiming at obtaining better performance than the individual modalities. However, when combining these modalities, the dimensionality of the feature space rapidly increases, yielding the well-known “curse of dimensionality”. As a consequence, in order to extract useful information from this data, one has to resort to feature selection (FS) techniques to lower the dimensionality of the learning space. In this paper, we assess the impact of FS techniques for silent speech data, in a dataset with 4 non-invasive and promising modalities, namely: video, depth, ultrasonic Doppler sensing, and surface electromyography. We consider two supervised (mutual information and Fisher’s ratio) and two unsupervised (mean-median and arithmetic mean geometric mean) FS filters. The evaluation was made by assessing the classification accuracy (word recognition error) of three well-known classifiers (knearest neighbors, support vector machines, and dynamic time warping). The key results of this study show that both unsupervised and supervised FS techniques improve on the classification accuracy on both individual and combined modalities. For instance, on the video component, we attain relative performance gains of 36.2% in error rates. FS is also
useful as pre-processing for feature fusion.