Theory of probability and Mathematical statistics

Asymptotic analysis of the statistical estimators of Shannon entropy of binary $s$-tuples

U. Yu. Palukha^ab, Yu. S. Kharin<sup>ab

a</sup> Belarusian State University, Minsk
^b Research Institute for Applied Problems of Mathematics and Informatics, Belarusian State University, Minsk

Abstract: The asymptotic probability distribution of the statistical estimate of Shannon entropy of $s$-tuples $\widehat{H}(s)$ and the asymp totic joint probability distribution of the statistical estimates of Shannon entropy of $s$- and $(s+1)$-tuples $\widehat{H}(s), \widehat{H}(s+1)$ for a uniformly distri buted random binary sequence with increasing length are found. It is proved that as the value of $s$ increase, the correlation coefficient of the statistical estimates of Shannon entropy of $s$- and $(s+1)$-tuples $\widehat{H}(s), \widehat{H}(s+1)$ tends to zero. The theoretical results are illustrated by computer experiments.

Keywords: Shannon entropy; binary sequence; statistical estimator; covariance; correlation coefficient.

UDC: 519.237.4