PATTERN RECOGNITION

On entropic criteria for feature selection in data analysis problems

Yu. A. Dubnov

Federal Research Center “Computer Science and Control” of Russian Academy of Sciences, Moscow, Russia

Abstract: The paper considers the problem of reducing the dimension of the feature space for describing objects in data analysis problems using the example of binary classification. The article provides a detailed overview of existing approaches to solving this problem and proposes several modifications. In which the dimensionality reduction is considered as the problem of extracting the most relevant information from the characteristic description of objects and is solved in terms of the Shanon's entropy. To identify the most significant features information criteria such as crossentropy, mutual information and Kullback-Leibler divergence are used.

Keywords: dimentionality reduction, feature selection, classification, entropy.

DOI: 10.14357/20718632180205

Bibliographic databases:

•