Creating and sharing knowledge for telecommunications

A comparison between Shannon’s, Renyi’s, and Tsallis Mutual Information for Feature Selection

Ferreira, A. ; Figueiredo, M. A. T.

A comparison between Shannon’s, Renyi’s, and Tsallis Mutual Information for Feature Selection, Proc Portuguese Conf. on Pattern Recognition - RecPad, Covilha, Portugal, Vol. --, pp. -- - --, October, 2014.

Digital Object Identifier: 0

Download Full text PDF ( 221 KBs)

Abstract
Feature selection (FS) techniques based on mutual information (MI) criteria have been used extensively in the last two decades. Since the seminal
work of Battiti in the context of neural networks, many FS filters have
been proposed. The results reported in the literature show the adequacy
of these methods, for many different problems. However, the vast majority of these methods rely on information-theoretic measures proposed
by Shannon. The extensions to Shannon’s work as proposed, namely, by
Renyi and Tsallis have received much less attention, in this context. This
paper has a two-fold objective: explore the use of Renyi’s and Tsallis
definitions of MI in the context of FS for supervised classification tasks;
bring some attention to the definitions proposed by Renyi and Tsallis.
The reported experimental results on public domain benchmarks, show
the adequacy of the three definitions for this purpose.